You can time-tag the source ip using a log forwarding profile built-in action.
Once the source is tagged, create an Address Group (Dynamic) (DAG) and set it to match the created tag.
You will then configure a Security Policy that will precede the current one being matched where the source is the DAG, and set the rule to Deny. The sources will remain tagged for the time lapse configured in the Log Forwarding profile built-in action, and after the time expires, they will be removed from the tag, therefore being matched again by the currently matched rule.
If you need instructions, I recently wrote an article on doing something similar to inhibit email alerts (retrigger timer). The article is not yet public because it is undergoing a revision process. If you need a copy please open a support case and ask for the case to be assigned to me. You can reference this post in the case.
... View more