URL filtering behavior when used in conjunction with web proxy

[lunaticitizen]

Hello guys,

I'll make it as short as possible. The summary of our system is below.

• Client PCs use must use an internal web proxy ("Proxy" in the diagram) when they want to access the internet.
• The internal web proxy then forwards the proxy connection to an external proxy. ("WSS" in the diagram)
• Palo Alto's ("PA" in the diagram) URL filtering is used to prevent users from accessing certain websites. The profile is applied to the policy controlling connections from client PCs to the Palo Alto firewall. It's (1) in the diagram.
• SSL decryption is not enabled.

We observed the following.

(A) Attempting access to restricted, plain HTTP-based websites will display the Palo Alto's standard response page. We also observed that the only transmission that occurred is transmission (1) in the diagram below.

(B) Attempting access to restricted, HTTPS-based websites will display the web browser standard error page "The connection is reset", but when we analyzed the log files of the internal web proxy and the external web proxy, we could also see the transmission log from the client PCs.

My question: is observation (B) an expected behavior with Palo Alto firewalls? We had expected the same behavior as observation (A) (i.e. we had expected the transmissions (2)(3)(4)(5) wouldn't have had occurred)

TIA for your time.

           +-----+     +---------+
           | WSS | --> | Website |
           +-----+ (5) +---------+
              ↑
        +------------+
        |The Internet|
        +------------+
              ↑(4)
            +----+ (3)  +-------+
            |    | <--- |       |
+----+      | PA |      | Proxy |
| PC | ---> |    | ---> |       |
+----+ (1)  +----+ (2)  +-------+

[reaper]

Are you performing ssl decryption?

These are connections between 2 proxies, rather than a web server, so behavior may differ from "expected" as either proxy may introduce additional checks to prevent intermediary devices from injecting response pages in ssl

[lunaticitizen]

Hello @reaper 

No, we don’t perform SSL decryption. What we want to understand first is why, in case of HTTPS, the firewall seems to forward transmissions even though they should have been blocked in (1).

Thanks in advance.

[reaper]

Normally the firewall should also provide the same block page, but there are many (especially with the proxies in place) cases where this process may get interrupted

Is the handshake altered in any way by the proxies, are the encryption suites supported, is the sni or other certificate atttibutes maintained/legible.to the firewall, ...

Could you compare the session (packet for packet) between a client directly on the internet vs a client behind the firewall to the same site to see if there is any difference?

[lunaticitizen]

Hello @reaper,

Thanks for your reply.

Please let me get this thing straight first. In our case, shouldn’t the internal proxy NOT come into play at all? All transmissions should be blocked in (1), shouldn’t they?

We don’t really mind whether the block page is displayed or not, actually.

[reaper]

ok, so the TCP handshake needs to go through the entire chain of devices

once that is completed, the client will send out a client hello, server willreply swith a server hello and so on

it depends on the information contained in the client hello if the firewall will be able to identify the url category. if for some reason it is not able to do so, it will need to wait for the server hello to return

this is where the extra proxies could make alterations and further complicate the situation

a packetcapture should be able to help you figure out what packets are transmitted and received

[lunaticitizen]

Hello @reaper,

Thank you for your response.

I see. I thought the firewall identifies the URL (or just FQDN) from the first packet (after TCP handshakes) clients send to the internal web proxy. In my understanding it's either regular HTTP requests (HTTP GET, etc.) for plain HTTP, or HTTP CONNECT for HTTPS. Either of them is just plain HTTP so the firewall should be able to see the inside of the packet without any decryption (I've confirmed this by checking the log files).

In case of HTTPS, if the firewall somehow needs to inspects packets that comes after HTTP CONNECT then I can see that the proxies will complicate things.

I'll do some more packet captures and report back to you, although I'm afraid it would be next Monday (it's already Friday night here in Japan).

Thanks for your time.

[reaper]

the firewall will only see the HTTP CONNECT if you have decryption enabled

without decryption, the firewall only sees client hello, server hello, key exchange and then a bunch of encrypted payload

enjoy your weekend!