topic Re: tcp out of order in General Topics

tcp out of order

jdprovine — Wed, 12 Sep 2018 12:29:09 GMT

I am trying to diagnose why I am unable to access some server via ssh and the VPN. I have done a packet trace and the results were concerning tcp out of order and retransmission. So what is the best way to find out why?

Re: tcp out of order

reaper — Wed, 12 Sep 2018 13:15:09 GMT

That's a tricky one

If you're able to packet capture on both ends, that is a great start, as it will allow you to see how packets arrive and leave at each point

Are you running ssh through your vpn or independently from eachother ?

Have you tried accessing from other networks, as to rule out which 'end' of the line is introducing issues

Re: tcp out of order

jdprovine — Wed, 12 Sep 2018 13:23:46 GMT

@reaper

I am unable to run a packet capture on the server that is not responding since I do not have access to login to it, but I will suggest it to someone who does. My packet capture was done on the palo filtering for the IP was was coming from to the IP I was trying to ssh too.

What we are doing is logging into the VPN and then trying to ssh to the server. This VPN only had access to the PCi network alone, though I have tried to ssh to other server in this same network.

So tcp out of order and retransmission refer to assymetry, dropped packets, not able to reassembly the packets, uturn NAT,can it be a block from the recieving server? I tried to use the cli to get more information but not able see the particular session in question

Re: tcp out of order

reaper — Wed, 12 Sep 2018 13:32:48 GMT

out of order means packets are received in an unusual order (eg. 1,4,2,3,6,7,5)

usually, this is caused by 'something in the middle' that is sending packets left and right causing delay to some packets in respect to the other packets, or a severely saturated server/link

Re: tcp out of order

jdprovine — Wed, 12 Sep 2018 13:36:39 GMT

@reaper

Re: tcp out of order

reaper — Wed, 12 Sep 2018 13:50:03 GMT

can you clarify how you discovered packets were out of order, this may already help troubleshooting

-if you log on to the vpn, are other resources available normally, or is everything disrupted

-if you ping the firewall, is this responding normally and quickly

-if you ping the server is this responding normally and quickly

-is traceroute showing you any odd 'hops'

-are sessions not being created? or is there another reason sessions can't be determined?

-if you follow the packetcaptures, are packets displaced immediately or is this happening after the tcp handshake?

-what do the global counters tell you? only "out of order" or are there other counters

Re: tcp out of order

jdprovine — Wed, 12 Sep 2018 13:55:26 GMT

@reaper

I discovered the packets were out of order in the packet capture I did from the firewall

Ping the firewall from the VPN or from the server I am trying to ssh too

I beieve for security reasons the server is set to not allow pings

traceroute from the VPN correct?

not sure what how to see the specific global counters for that specific session

I believe session are being created but aging out

The only thing I see in the packet captures is tcp out of order and retransmission unless I am not looking in the right place and something about group sessions not being accepted

Re: tcp out of order

reaper — Thu, 13 Sep 2018 08:05:11 GMT

i got an article for that 😉 https://live.paloaltonetworks.com/t5/Featured-Articles/Getting-Started-Packet-Capture/ta-p/72069