How to trigger a "Response page" on Palo Alto NGFWs using URL filtering & Decryption

[PS007]

Hi All, 

I want to share my experience on very latest configuration I did in my company Firewall infra. With the genAI evolution & most of the companies trying to develop their own homegrown genAI app, it was becoming headache for my Cyber Risk team to track usage of 3rd party genAI apps (like ChatGPT, Gemini) & doing comparative analysis of traffic for external genAI vs internal genAI usage. Risk also wanted to deter colleagues from uploading company's data on unapproved sites, get logging visibility for personal files sharing, third party email & at  same time wanted effective user education in place with list of guidelines to use such apps with caution was very critical for them to issue warning/splash screen.

I discussed these challenges with Risk team and inspired with some hands-on SANS training and getting some skin in the game with my domain expertise came up with idea and implemented  a solution using via PANW firewalls "Response page" feature. 

Solution :

1. As part of requirement gathering you have to ask your customer (Security/Risk/SOC) to provide list of sites that requires users to accept the Risk T&C with authentication. Consider this as your Allow list & in later phases consider hosting this on EDL server (Of Course you need automation & don't want to update the list manually). 
2. Create a "custom" URL category considering it as Allow list by importing it in Firewall. 
3. Create a URL filtering profile calling out the above specific custom URL category with action of "continue".  (PAN firewall intercept the c2s flow with SSL Decryption in Forward Proxy mode discussed later). Rest of the custom URL categories should be set to action "none" & pre-defined URL categories should be set to action "block". Action of “none” means firewall URL filtering will not check that category for any action , which is expected in this solution. 
4. Based on your choice either create a Security Profile group and use the above URL filtering profile along with other essential  profiling for AV,Vulnerability management, File Blocking etc. or simply you can call them out individually in security policy (aka rule). 
5. Turn ON TLS/SSL decryption- Intercepting TLS/SSL is critical here so that Firewall can provide proper response page with different messaging for genAI & File Sharing sites. Firewall will read SNI or check the certificate to determine the URL categorization. Since the custom URL categorization takes precedence over pre-defined URL category, firewall will match it to your custom URL category listed sites & take action defined.  
6. Above can be done by creating an SSL Decryption policy to decrypt by calling out custom URL category you created. Even if SSL Decryption is turned ON, traffic remains encrypted at firewall level and gets split into two flows :

   (i) Between the Browser/client and the Firewall- serve up the continue page over http:6080 ( log action : block-continue explained later)

   (ii) Between the Firewall and the Destination website- serve as the direct interaction between user and website after “continue” (although firewall on behalf of client/user machine is talking to server(website). In such interaction firewall can log additional fields like uploaded file names, user IP, user-id, timestamp, app-id etc. Later you can see related logs in “Data-filtering” logging section for such traffic.

7. Now create a security policy/ rule and write it to match/add custom URL categories defined. With Least privilege access in mind restrict it with source, destination  fields and it is highly recommended to use source User-ID field with integration with your LDAP/AD for group mapping and call out User groups (may be Risk based like External workers, contractors, Finance group etc.).  You should be using Security profile group created in step 4  in the above policy under profile settings group which have URL filtering profiles with action “continue” for custom  URL category.
8. Ensure you are enabling Logging on this  Security & Decryption rule forwarding to your SIEM or Panorama if using it in Logger mode. This will also help you later to create On-demand or scheduled reports for number of users/session count/packet in & out against  3rd party or gen AI filesharing sites as you can now easily set filter on matching traffic with action "continue" & send it to your Risk/SOC for IH,IR via scheduled emails sent from your PANW firewall with usage analysis based on Top talkers, bytes etc. 
9. As a last step design an HTML script to give warning/splash screen with T&C message defined. Import the script in txt format under Device> Response pages > "URL Filtering Continue and Override page". This ensures you will display appropriate or discrete text/message when users visit websites subjected to “continue” response. Please reach out to me if you need help with HTML script.
10. Test the solution by visiting the website part of your custom URL category and follow URL filtering log with filter as action "block-continue" & then "continue" for that site. Action “block-continue” will appear once when user is presented with continue page. Once, continue is clicked rest of the logs will be with action “continue” until URL timeout value is reached or counter resets if user machine IP gets changed. URL Continue Timeout (min) can be set under Setup>Content-ID> URL filtering settings. Ideally you may want to change it to at least 24 hrs/1440 min or more if your users complains the warning page appear frequently. Default value is 5 min. 

Considerations : 

• After implementing & testing this solution you will notice that URL address bar has unfriendly address over port 6080. This pertains to flow Step 6 (i) explained above and is seen because :The "Continue Page" requires a user interaction (clicking the continue button) which needs to be served up by a web service. The ports 6080-6082/TCP are web service that the firewall is using as a landing spot for that page. The URL in the client browser unfortunately cannot be changed as it contains all the needed information for the firewall to track the redirection session to a specific user. This is a required step to achieve this functionality and cannot be altered. When the user clicks the "Continue" button the acknowledgment is sent to the webservice and the originally blocked firewall session is now allowed.

• Although it is evident while presenting continue page an unencrypted traffic over http:6080 is seen but at this stage no user payload or header data is exchanged. Till this time primary function is to present continue page with warning message & traffic still remain encrypted at firewall level while splitting in two flows (i) & (ii). Here is the KB explaining how this works - Understanding this redirection 

Please feel free to share comment, feedback.