Why Did Strict IP Address Check Break this VPN?

[ms.jzam]

We have been working with TAC to find the cause of this issue where FTP client could no longer upload to external companies FTP server over the VPN tunnel.  After many days, we started a packet filter on the Public Internet (WAN) interface, which is a different zone from the tunnel interface, and were still seeing drops due to "flow_dos_pf_strictip".  We had previously disabled the zone_protection policy that was applied to the tunnel interface zone, and even though we did not see drops, the uploads were failing.  When we finally did that packet filter on the WAN interface we saw the drops again due to the same reason "flow_dos_pf_strictip" and decided to remove the zone protection policy from the WAN interface.  BOOM!  FTP uploads succeed.  Combing through the policy I saw the strict ip check option and removed it, then pushed the policy overwriting our revert.  Everything still good.

We couldn't find the configuration change that put that feature in place (wish that function worked better in the Palo, or just knew how to use it better!) and reading the KB pages for that feature, I'm not sure why it was causing our traffic to be dropped.

Any help would be very much appreciated!

[BPry]

@ms.jzam,

What IP address were you sending to from within that tunnel? If I had to harbor a guess, the IP in question is technically not valid if you follow RFC 1918. 

[ms.jzam]

@BPry

IKE Gateway Local IP Address 216.1.x.x Peer IPA 199.79.x.x

IPSec Tunnel used Proxy ID which was Local 172.16.2x.x (Internal Server IP of 10.x.x.x was NATd) Remote 128.1.2xx.x

[ms.jzam]

BUMP

[jdprovine]

@ms.jzam

Same here I had to take zone protection off of my untrust zone cause it was breaking my VPN. I will have to try deselecting strict IP check and see if it fixes my issue

[ms.jzam]

@jdprovine  Let us know what you find!  If it does fix the issue, that's a much better place to be than disabling the whole ZP policy.

Hopefully we can get some progress on understanding the root cause behind the issue.

[jdprovine]

@ms.jzam

I have a ticket in with TAC about it and so far they told me that it wasn't likely the cause of my VPN breaking though yesterday I was able to repeat the issue by merely enabling Zone protection. I would be very interested to see why the strict IP address cause the issue. TAC also advised me to turn on spoofing IP address, which did not work.

[ms.jzam]

@jdprovine

I would reccommend taking the same steps that I did in order to get a clue as to the cause.  Include your VPN tunnel network/interface in a packet capture filter, and also your WAN interface where the VPN traffic is coming into.  

clear logs and packet filter

debug dataplane packet-diag clear all

debug dataplane packet-diag clear log log

enable debug

debug dataplane packet-diag set log feature flow basic

debug dataplane packet-diag set log on

debug dataplane packet-diag set log off

debug dataplane packet-diag aggregate-logs

review logs

less dp-log pan_packet_diag.log

clear your vpn tunnels and see if interesting traffic reestablishes them

clear vpn ike-sa 

clear vpn ipsec-sa

check and configure your packet filter

debug dataplane packet-diag show setting 

*was easier for me to configure the packet filter in the WEB-UI

check for vpn sesssions

show session all filter source X destination X

show vpn flow

*pay attention for encap and decap byte increase

check the details of the specific traffic session

show session id X

show results of the packet filter

show counter global filter packet-filter yes delta yes

There is probably a better order to running these commands, but this is everything I used to try and get to a resolution.

[BrianRa]

There may be something in your Zone Protection Profiles that is configured other than what you are intending.  For example out of order packets or asymetric routing (just examples).

From the CLI you can type out these commands and it will give you a good config dump of what your profiles look like.  It may help to sanitize them and paste them for people to take a look at.

pa-firewall> set cli config-output-format set   (this sets the way the output will be displayed and only lasts the current session)

pa-firewall> configure

pa-firewall# show network profiles zone-protection-profile  (this will display the profiles in line format, not xml)

I try to use this to make sure what is being put in is actually doing what I want (if anyone remembers back in the cisco "gui" days *cough*pix*cough*).

[ms.jzam]

@BrianRa

ran the commands as you indicate but received no output:

(active)> 
(active)> set cli config-output-format set
(active)> configure
Entering configuration mode
[edit]                                                                                                                                            
(active)# show network profiles zone-protection-profile 
  <name>   <name>
  |        Pipe through a command
  <Enter>  Finish input

(active)# show network profiles zone-protection-profile 
set network profiles zone-protection-profile 
[edit]                                                                                                                                            
(active)# show network profiles zone-protection-profile ZP_DEFAULT_OUT
[edit]                                                                                                                                            
(active)#

[BrianRa]

@ms.jzam do you currently have the profiles running on the firewall?  If not they will not show up in the current configuration.

Also if you are pushing from Panorama then they will not show up on the local firewall configuration (sorry I didn't think to mention this before).  These configs can be shown on the local firewall however they only show as xml (there is no option to change this).

pa-firewall> show config pushed-template

pa-firewall> show config pushed-shared-policy

From Panorama CLI you can view these as well but it is more convoluted to get to a specific firewall config.

Panorama> set cli config-output-format set

Panorama# show device-group <pa-firewall>

Panorama# show template <pa-firewall> config network profiles zone-protection-profile

[ms.jzam]

@BrianRa  yes it was because of Panorama!  Thanks!

[ms.jzam]

@jdprovine

Anything new on this?

[jdprovine]

@ms.jzam

I am trying to find a maintenance window to test and collect logs and do a packet capture. I am hoping maybe i will get luck tomorrow morning though unlike other places I have worked most users are on the VPN during the work day instead of the off shift or I would have done it by now LOL 😛

[jdprovine]

@ms.jzam

Amazing the VPN started to work again when I deselected Strict IP Address Check.