New Advanced URL Filtering Category: Scanning Activity

[almraza]

We intend to introduce a new category called “Scanning Activity” under Advanced URL Filtering. 

ACTION: Your action is required. By default, we set the “Scanning Activity” category to “Block” mode for the default profile only. If you have multiple URL Filtering profiles, we recommend that you change the default action to “Block” for this category in each of your profiles.

How is Scanning Activity Defined?

Adversaries are increasingly taking advantage of infected hosts to scan a network for vulnerabilities and launch targeted attacks. Additionally, attackers frequently include such probing activities in their malicious campaigns to carry out attacks on a network. Palo Alto Networks defines these scanning and probing tactics as “Scanning Activity” and are considered to be indicators of compromise.

Will the “Scanning Activity” category be visible across all versions of PAN-OS?

Yes, the Scanning Activity category will be visible across all supported PAN-OS versions. However, it is functional only for firewalls running PAN-OS 9.1 and later versions and requires  an Advanced URL filtering license. 

When will the “Scanning Activity” category be available?

The “Scanning Activity” category will be visible on the administrator management console after you install the content release scheduled for July 11, 2023 (or a later version). However, we will not begin publishing URLs to this new category until November 28, 2023. 

When will the “Scanning Activity”  category be functional?

Starting November 28, 2023, Palo Alto Networks will start publishing URLs that are categorized as “Scanning Activity.” Please ensure that your Security policy rules are configured to account for this new category.

What is the recommended action for the “Scanning Activity” category?

Scanning activity is an indicator of compromise that can pose a serious threat to users and businesses. Therefore, we recommend that you keep the default action for this category set to “Block.”

Note: The “Scanning Activity” category action is set to “Block” only for the default profile. If you have multiple URL Filtering profiles, an administrator must update the default action to “Block” for this category in each of your profiles. This requirement applies to all supported versions of PAN-OS software.

Why is the new "Scanning Activity" category missing under my URL filtering profiles?
The 'Scanning Activity' category is only available to customers with the content release version 8729 and above. It will not be visible in previous versions. To take advantage of this new category, customers are required to update to the appropriate content release version.

What happens if my NGFW is still using the content version below 8729?
For customers using content versions below 8729, published URLs in this new category will not be categorized under 'Scanning Activity' and will instead be classified as 'Unknown'. If the NGFW's 'Unknown' category is set to 'block,' these URLs can be blocked accordingly.

How do I avoid disruptions during the scheduled vulnerability scanning/ penetration testing within my network?

The Scanning Activity category may detect and block sanctioned penetration testing or scheduled scanning traffic from scanner services, which are routinely run to comply with best practices. To prevent this traffic from being flagged and blocked by the Scanning Activity category, we recommend whitelisting the IP addresses of the scanners generating this traffic within the security policies.

For more information, please see the KB article: Firewall setting to reduce the FW interference for pen-test on a resource behind the FW.

What action should I take when the “Scanning activity” category is triggered?
Scanning activities within your network is an indicator of compromise that can pose a significant threat to your users and business. Therefore, we recommend the following:

• Ensure that the category is set to “Block.”
• Check the source IP address of users who generate this type of traffic under the URL Filtering log to isolate the host.

Note: Scanning activity detention is agnostic if the processed traffic is from ingress or egress. If the source IP address of the scan does not belong to your network, please check if the URL filtering profile is being applied to the ingress traffic under your security policy.

What is the Palo Alto Networks test URL for Scanning Activity?

http://urlfiltering.paloaltonetworks.com/test-scanning-activity

Additional Information

For more information on best practices when managing Advanced URL Filtering categories, please read our 

• URL Filtering Category Recommendations 
• Complete List of PAN-DB URL Filtering Categories

[ElliotM]

There is conflicting information being provide by Palo Alto here. This article states URLs will be added to the new category in Oct however the 'Palo Alto Networks Content Update' emails states November.

Which is it ?

[DavidCHW]

your links is dead ?

http://urlfiltering.paloaltonetworks.com/test-scanning-activity

[almraza]

Hi ElliotM, thanks for bringing it to our attention. Correct date is Oct 11th. We are correcting the content RN.

[UtkarshB]

When will the “Scanning Activity” category be available?

The “Scanning Activity” category will be visible on the administrator management console after you install the content release scheduled for July 11, 2023 (or a later version). However we will not begin publishing URLs to this new category until October 11, 2023. 
####################

Do you know when this will be available.
I want to set action as block but it is currently not available as I check the update on July 11

[almraza]

It will be included with today's content release by the EOD. 

[AllenBrown]

Thanks for the update sir.

[UtkarshB]

Thanks. 
I was able to see 'scanning-activity' is available by EOD GMTime on July 11, 2023.

[jeffchenes]

How does this new category "Scanning-activity" compare to the threat category and type SCAN?

[UmerSalahuddin]

Customer is asking for a SOP document in order to make changes in their Custom URL filtering profile for the Artificial intelligence and scanning activity categories. We want to implement these actions.

ACTION: Your action is required. By default, we set the “Scanning Activity” category to “Block” mode for the default profile only. If you have multiple URL Filtering profiles, we recommend that you change the default action to “Block” for this category in each of your profiles.

ACTION: Action will be required. The “Artificial Intelligence” category action is set to “Alert” only for the default profile. If you have multiple URL Filtering security profiles, it is recommended you change the default action to “Alert” for each of these profiles for better visibility and control.

[kumarash]

I hope this will not impact on tenable traffic?

[EdwardDiaz]

In our interface. we don't even see anything that says "Advanced URL Filtering". We have an active Advanced URL Filtering subscription, on PanOS 9.1.14-h4  

I was able to find a "scanning-activity" under Objects > Security Profiles >  URL Filtering > [clicking on a filter] > Categories > Scroll down to "Pre-Defined Categories"

Hope this is the correct location to set the block before we get the big zap in the sky...

[UmerSalahuddin]

That is correct. It is found in "Pre-Defined Categories"

[Sanjith-Nimshad]

Are there any URLs from this category that can be tested after making it to block?

[Kenber]

Does anyone have any information on how to test for this change or could it cause any problems?

[EdwardDiaz]

As far as I can tell, the only test to verify that the blocked URL categories are working is going to the page Palo ALto setup and marked as "scanning-activity":

http://urlfiltering.paloaltonetworks.com/test-scanning-activity

[tcervellione]

Why does it say "TBD" on content update emails, but the verbiage in this article hasn't changed, and the Oct. 11th scheduled date for release remains the same?

That's very confusing for a new URL category that we're recommending block actions and potential production risk for. Can we clarify this quickly please? 

[LeeFrancis2]

I echo the request for urgent clarification

Many end user organisations will have internal security scanning systems. There is also no information on the signature or proflle of the traffic that wil be matched. I tthink PAN should provide this detail to re-assure customers that internal network monitoring systems that may ping/snmp ranges of IP addresses will not be falsely detected.

This documents states:

Note: While URL filtering typically does not see inbound traffic, our detection technology can identify scanning activities regardless of the traffic direction. Therefore, if the source IP address does not belong to your network, check if the URL filtering profile is assigned to the inbound interface.

Please advise ASAP where 'assignment to an interface' is set and how to check this. It is not at all obvious.

KR,

Lee

[MikeGeo]

Couple questions because the above information is not clear:

1. Exactly "what" type of Outbound Traffic will this new URL Category detect?

1a. Internet Outbound URLs of attempts to "phone home" from an infected machine?

1b. Will this impact Internal Network Scanners connecting back to their Cloud? Like Tenable On-Premise Servers getting updates?

2. Exactly "what" type of Inbound Traffic will this new URL Category Detect? We were previously told by Palo Support that URL Filtering only applies to Outbound Traffic which conflicts with the statement below from the article.

2a. If this applies to Internet Inbound Traffic, are you recommending that a URL Category now be assigned to Internet Inbound Policies?

Note: While URL filtering typically does not see inbound traffic, our detection technology can identify scanning activities regardless of the traffic direction. Therefore, if the source IP address does not belong to your network, check if the URL filtering profile is assigned to the inbound interface.

[Ridwaan]

The test link only works for port 80 (http) traffic and not for 443 (https) is this expected behavior ?

[ErinWest]

The test link is not blocked for us because the full path is not showing within the logs, only the main site which we use for recategorization and that is allowed as computer/internet. Is there any other test url's we can use? Any list of url's they added yet? 

[aleksandar.astardzhiev]

Hi @Ridwaan ,

This means you are not appling SSL decryption for this traffic. In that case it is expected to work only with plain-text HTTP.

Hey @ErinWest ,

It looks like you also don't decrypt this traffic. Without SSL decryption firewall is not capable of inspecting the full URL path, because thi is part of the HTTP headers that are exchanges after SSL encryption is negotiated.

When traffic is SSL encrypted firewall will have visibility up to the SSL/TLS negotiation where the SNI (server name indicator) is indicating which hostname the client is trying to reach.

[ErinWest]

So we set up for decryption on this category and it is still not blocked. And yes other categories do get our block page so we know that decryption is working. 

[BPry]

@ErinWest,

If you're not getting the entire URL included in your log message really the only possible way for that to happen is that you aren't decrypting the traffic. There's a couple of potential reasons that this could be happening if you intend to decrypt this traffic, but the end of it is really the same, the traffic isn't being decrypted. You can verify this via your logs by adding the decrypted column.

It's possible that you have urlfiltering.paloaltonetworks.com in a custom category that is overriding the scanning-activity categorization and thus allowing the traffic. If you're additionally selective in what you decrypt and utilize explicit categorization for decrypting traffic instead of explicit categorization for excluding the traffic from decryption, it would also explain both behaviors that you're seeing. 

I'd look through your configuration and verify that you aren't accidently overriding that categorization and causing the traffic to both be allowed, and also not be decrypted as you're expecting. 

[ErinWest]

So the thing is we are decrypting other content categories in the same rule, like hacking sites are blocked etc. We have a case open but they haven't found why this particular one is not being blocked when others are within the same decryption rule. We do not have that PA url on an explicit allow list either. Will just have to sort it out!