Some webs not working

[soporteseguridad]

Hi,

We have a PA-3050 with PanOS 8.0.5. We cant access to these webs https://www.metromadrid.es and https://www.ing.es

We receive a timeout and we see a RST from Palo Alto to web server.

We had this problem in 8.0.4 and we upgraded to 8.0.5 but the problem persists.

This issue breaks my mind, can someone give me any idea???

[kiwi]

Hi @soporteseguridad,

If there's an algorithm incompatibility then you can verify this in the PCAPs. 

The PCAP will show you the Cipher Suite the servers sends back.

The following information is an extract taken from https://www.ssllabs.com/ssltest where you can analyse server SSL reports.

https://www.metromadrid.es supports the following cipher suites :

# TLS 1.2 (suites in server-preferred order)
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030)   ECDH secp384r1 (eq. 7680 bits RSA)   FS	256
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f)   ECDH secp384r1 (eq. 7680 bits RSA)   FS	128
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (0xc028)   ECDH secp384r1 (eq. 7680 bits RSA)   FS	256
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014)   ECDH secp384r1 (eq. 7680 bits RSA)   FS	256
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (0xc027)   ECDH secp384r1 (eq. 7680 bits RSA)   FS	128
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (0xc013)   ECDH secp384r1 (eq. 7680 bits RSA)   FS	128
TLS_RSA_WITH_AES_256_GCM_SHA384 (0x9d)	256
TLS_RSA_WITH_AES_128_GCM_SHA256 (0x9c)	128
TLS_RSA_WITH_AES_256_CBC_SHA256 (0x3d)	256
TLS_RSA_WITH_AES_256_CBC_SHA (0x35)	256
TLS_RSA_WITH_AES_128_CBC_SHA256 (0x3c)	128
TLS_RSA_WITH_AES_128_CBC_SHA (0x2f)	128
# TLS 1.1 (suites in server-preferred order)
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014)   ECDH secp384r1 (eq. 7680 bits RSA)   FS	256
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (0xc013)   ECDH secp384r1 (eq. 7680 bits RSA)   FS	128
TLS_RSA_WITH_AES_256_CBC_SHA (0x35)	256
TLS_RSA_WITH_AES_128_CBC_SHA (0x2f)	128
# TLS 1.0 (suites in server-preferred order)
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014)   ECDH secp384r1 (eq. 7680 bits RSA)   FS	256
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (0xc013)   ECDH secp384r1 (eq. 7680 bits RSA)   FS	128
TLS_RSA_WITH_AES_256_CBC_SHA (0x35)	256
TLS_RSA_WITH_AES_128_CBC_SHA (0x2f)	128

https://www.ing.es supports the following cipher suites :

# TLS 1.2 (suites in server-preferred order)
TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 (0x9f)   DH 1024 bits   FS   WEAK	256
TLS_DHE_RSA_WITH_AES_256_CBC_SHA (0x39)   DH 1024 bits   FS   WEAK	256
TLS_RSA_WITH_AES_256_GCM_SHA384 (0x9d)	256
TLS_RSA_WITH_AES_256_CBC_SHA256 (0x3d)	256
TLS_RSA_WITH_AES_256_CBC_SHA (0x35)	256
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030)   ECDH secp256r1 (eq. 3072 bits RSA)   FS	256
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (0xc028)   ECDH secp256r1 (eq. 3072 bits RSA)   FS	256
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014)   ECDH secp256r1 (eq. 3072 bits RSA)   FS	256
TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA (0x16)   DH 1024 bits   FS   WEAK	112
TLS_RSA_WITH_3DES_EDE_CBC_SHA (0xa)   WEAK	112
TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA (0xc012)   ECDH secp256r1 (eq. 3072 bits RSA)   FS   WEAK	112
TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (0x9e)   DH 1024 bits   FS   WEAK	128
TLS_DHE_RSA_WITH_AES_128_CBC_SHA (0x33)   DH 1024 bits   FS   WEAK	128
TLS_RSA_WITH_AES_128_GCM_SHA256 (0x9c)	128
TLS_RSA_WITH_AES_128_CBC_SHA256 (0x3c)	128
TLS_RSA_WITH_AES_128_CBC_SHA (0x2f)	128
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f)   ECDH secp256r1 (eq. 3072 bits RSA)   FS	128
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (0xc027)   ECDH secp256r1 (eq. 3072 bits RSA)   FS	128
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (0xc013)   ECDH secp256r1 (eq. 3072 bits RSA)   FS	128
# TLS 1.1 (suites in server-preferred order)
TLS_DHE_RSA_WITH_AES_256_CBC_SHA (0x39)   DH 1024 bits   FS   WEAK	256
TLS_RSA_WITH_AES_256_CBC_SHA (0x35)	256
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014)   ECDH secp256r1 (eq. 3072 bits RSA)   FS	256
TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA (0x16)   DH 1024 bits   FS   WEAK	112
TLS_RSA_WITH_3DES_EDE_CBC_SHA (0xa)   WEAK	112
TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA (0xc012)   ECDH secp256r1 (eq. 3072 bits RSA)   FS   WEAK	112
TLS_DHE_RSA_WITH_AES_128_CBC_SHA (0x33)   DH 1024 bits   FS   WEAK	128
TLS_RSA_WITH_AES_128_CBC_SHA (0x2f)	128
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (0xc013)   ECDH secp256r1 (eq. 3072 bits RSA)   FS	128
# TLS 1.0 (suites in server-preferred order)
TLS_DHE_RSA_WITH_AES_256_CBC_SHA (0x39)   DH 1024 bits   FS   WEAK	256
TLS_RSA_WITH_AES_256_CBC_SHA (0x35)	256
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014)   ECDH secp256r1 (eq. 3072 bits RSA)   FS	256
TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA (0x16)   DH 1024 bits   FS   WEAK	112
TLS_RSA_WITH_3DES_EDE_CBC_SHA (0xa)   WEAK	112
TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA (0xc012)   ECDH secp256r1 (eq. 3072 bits RSA)   FS   WEAK	112
TLS_DHE_RSA_WITH_AES_128_CBC_SHA (0x33)   DH 1024 bits   FS   WEAK	128
TLS_RSA_WITH_AES_128_CBC_SHA (0x2f)	128
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (0xc013)   ECDH secp256r1 (eq. 3072 bits RSA)   FS	128

Cheers !

-Kiwi.

[soporteseguridad]

I think the RST is sent before SSL negotiation. You can see it in the capture.

[mgarg]

This does not seem to be an SSL issue because we do not see even the 3-way handshake completing.

For the first packet from source port 57901, there is not corresponding syn ack. What is see is an 'Ack' packet from the server, which is something the client would not be expecting and thus resets the connection in the following packet. 

Take simultaneously pcap on firewall and client and compare packets for one particular stream.

Also, you can check global counters. On the firewall set the filter as source 10.160.74.142 and destination as 193.41.234.29.

Turn on the filter, access the website and in the cli type "show counter global filter packet-filter yes delta yes" 3 -4 times.

If the firewall is sending rst due to some reason, then counters should show some reason. 

Post your results.

[soporteseguridad]

Hi, i tried with web www.metromadrid.es. I attch the screenshots:

This is the pcap. IPs:

10.160.74.142: My ip PC

188.87.x.x: NAT publi IP Palo Alto

185.89.60.64: Web server metromadrid.es

This is the global counter reset using filter source my IP to web server and return:

What could be the reason for this RST?? thanks

[mgarg]

in the global counters i see a warning "tcp_drop_out_of_wnd - out-of-window packets dropped". As per the packet captures the client seems to be resetting the connection because the 3-way handshake is not completing and the weird part is that it sends retransmission packets after sending rst packets. 

Anyway, here is an explanation as to why the firewall is dropping the packet and you can test if the command that is given below to check if it works ( Just a way to confirm the problem and not a solution).

tcp_drop_out_of_wnd - out-of-window packets dropped

The Palo Alto Networks Firewall creates a sliding sequence window starting with the original ACK (the window size is based on the type of traffic within the session). It is expected that the packet sequence numbers within the current session reside within this sliding window. This window adjusts with the type of traffic and whenever new ACK messages are received. The default behavior on the device is to drop packets when sequence numbers are outside this window.

This feature can be added to the config file and applied with a commit, which is required to override the default behavior after rebooting:

> configure

# set deviceconfig setting tcp asymmetric-path < bypass|drop >

# commit

I think the packets that you have captured are client side. It would be more helpful if you can capture the packet on the firewall as well for receive, transmit and drop stage. That way you can check what response is received from the server before the firewall drops the packet.

[mhannan313]

Facing a similar issue here: https://live.paloaltonetworks.com/t5/next-generation-firewall/packets-retransmission-captured-in-pac...

Any recommendations?