You can't exclude in an address or address-group object. If you want this feature you would need to reach out to your SE and get a feature request put together or have your vote added to an existing request.
You are correct that negate-source and negate-destination will negate anything specified and match everything else.
Click Accept as Solution to acknowledge that the answer to your question has been provided.
The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!
These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!
The LIVEcommunity thanks you for your participation!