topic Re: Why Did Strict IP Address Check Break this VPN? in General Topics

Why Did Strict IP Address Check Break this VPN?

ms.jzam — Wed, 24 Jan 2018 21:25:44 GMT

We have been working with TAC to find the cause of this issue where FTP client could no longer upload to external companies FTP server over the VPN tunnel. After many days, we started a packet filter on the Public Internet (WAN) interface, which is a different zone from the tunnel interface, and were still seeing drops due to "flow_dos_pf_strictip". We had previously disabled the zone_protection policy that was applied to the tunnel interface zone, and even though we did not see drops, the uploads were failing. When we finally did that packet filter on the WAN interface we saw the drops again due to the same reason "flow_dos_pf_strictip" and decided to remove the zone protection policy from the WAN interface. BOOM! FTP uploads succeed. Combing through the policy I saw the strict ip check option and removed it, then pushed the policy overwriting our revert. Everything still good.

We couldn't find the configuration change that put that feature in place (wish that function worked better in the Palo, or just knew how to use it better!) and reading the KB pages for that feature, I'm not sure why it was causing our traffic to be dropped.

Any help would be very much appreciated!

Re: Why Did Strict IP Address Check Break this VPN?

BPry — Wed, 24 Jan 2018 22:06:51 GMT

@ms.jzam,

What IP address were you sending to from within that tunnel? If I had to harbor a guess, the IP in question is technically not valid if you follow RFC 1918.

Re: Why Did Strict IP Address Check Break this VPN?

ms.jzam — Thu, 25 Jan 2018 15:32:10 GMT

@BPry

IKE Gateway Local IP Address 216.1.x.x Peer IPA 199.79.x.x

IPSec Tunnel used Proxy ID which was Local 172.16.2x.x (Internal Server IP of 10.x.x.x was NATd) Remote 128.1.2xx.x

Re: Why Did Strict IP Address Check Break this VPN?

ms.jzam — Mon, 29 Jan 2018 23:43:02 GMT

BUMP

Re: Why Did Strict IP Address Check Break this VPN?

jdprovine — Tue, 30 Jan 2018 15:55:48 GMT

@ms.jzam

Same here I had to take zone protection off of my untrust zone cause it was breaking my VPN. I will have to try deselecting strict IP check and see if it fixes my issue

Re: Why Did Strict IP Address Check Break this VPN?

ms.jzam — Tue, 30 Jan 2018 16:02:20 GMT

@jdprovine Let us know what you find! If it does fix the issue, that's a much better place to be than disabling the whole ZP policy.

Hopefully we can get some progress on understanding the root cause behind the issue.

Re: Why Did Strict IP Address Check Break this VPN?

jdprovine — Tue, 30 Jan 2018 16:06:00 GMT

@ms.jzam

I have a ticket in with TAC about it and so far they told me that it wasn't likely the cause of my VPN breaking though yesterday I was able to repeat the issue by merely enabling Zone protection. I would be very interested to see why the strict IP address cause the issue. TAC also advised me to turn on spoofing IP address, which did not work.

Re: Why Did Strict IP Address Check Break this VPN?

ms.jzam — Tue, 30 Jan 2018 16:59:54 GMT

@jdprovine

I would reccommend taking the same steps that I did in order to get a clue as to the cause. Include your VPN tunnel network/interface in a packet capture filter, and also your WAN interface where the VPN traffic is coming into.

clear logs and packet filter

debug dataplane packet-diag clear all

debug dataplane packet-diag clear log log

enable debug

debug dataplane packet-diag set log feature flow basic

debug dataplane packet-diag set log on

debug dataplane packet-diag set log off

debug dataplane packet-diag aggregate-logs

review logs

less dp-log pan_packet_diag.log

clear your vpn tunnels and see if interesting traffic reestablishes them

clear vpn ike-sa

clear vpn ipsec-sa

check and configure your packet filter

debug dataplane packet-diag show setting

*was easier for me to configure the packet filter in the WEB-UI

check for vpn sesssions

show session all filter source X destination X

show vpn flow

*pay attention for encap and decap byte increase

check the details of the specific traffic session

show session id X

show results of the packet filter

show counter global filter packet-filter yes delta yes

There is probably a better order to running these commands, but this is everything I used to try and get to a resolution.

Re: Why Did Strict IP Address Check Break this VPN?

BrianRa — Tue, 30 Jan 2018 17:27:29 GMT

There may be something in your Zone Protection Profiles that is configured other than what you are intending. For example out of order packets or asymetric routing (just examples).

From the CLI you can type out these commands and it will give you a good config dump of what your profiles look like. It may help to sanitize them and paste them for people to take a look at.

pa-firewall> set cli config-output-format set (this sets the way the output will be displayed and only lasts the current session)

pa-firewall> configure

pa-firewall# show network profiles zone-protection-profile (this will display the profiles in line format, not xml)

I try to use this to make sure what is being put in is actually doing what I want (if anyone remembers back in the cisco "gui" days *cough*pix*cough*).

Re: Why Did Strict IP Address Check Break this VPN?

ms.jzam — Tue, 30 Jan 2018 17:35:04 GMT

@BrianRa

ran the commands as you indicate but received no output:

(active)> 
(active)> set cli config-output-format set
(active)> configure
Entering configuration mode
[edit]                                                                                                                                            
(active)# show network profiles zone-protection-profile 
  <name>   <name>
  |        Pipe through a command
  <Enter>  Finish input

(active)# show network profiles zone-protection-profile 
set network profiles zone-protection-profile 
[edit]                                                                                                                                            
(active)# show network profiles zone-protection-profile ZP_DEFAULT_OUT
[edit]                                                                                                                                            
(active)#

Re: Why Did Strict IP Address Check Break this VPN?

BrianRa — Tue, 30 Jan 2018 17:56:57 GMT

@ms.jzam do you currently have the profiles running on the firewall? If not they will not show up in the current configuration.

Also if you are pushing from Panorama then they will not show up on the local firewall configuration (sorry I didn't think to mention this before). These configs can be shown on the local firewall however they only show as xml (there is no option to change this).

pa-firewall> show config pushed-template

pa-firewall> show config pushed-shared-policy

From Panorama CLI you can view these as well but it is more convoluted to get to a specific firewall config.

Panorama> set cli config-output-format set

Panorama# show device-group <pa-firewall>

Panorama# show template <pa-firewall> config network profiles zone-protection-profile

Re: Why Did Strict IP Address Check Break this VPN?

ms.jzam — Tue, 30 Jan 2018 18:04:46 GMT

@BrianRa yes it was because of Panorama! Thanks!

Re: Why Did Strict IP Address Check Break this VPN?

ms.jzam — Thu, 01 Feb 2018 17:40:21 GMT

@jdprovine

Anything new on this?

Re: Why Did Strict IP Address Check Break this VPN?

jdprovine — Thu, 01 Feb 2018 17:44:09 GMT

@ms.jzam

I am trying to find a maintenance window to test and collect logs and do a packet capture. I am hoping maybe i will get luck tomorrow morning though unlike other places I have worked most users are on the VPN during the work day instead of the off shift or I would have done it by now LOL 😛

Re: Why Did Strict IP Address Check Break this VPN?

jdprovine — Fri, 02 Feb 2018 13:24:45 GMT

@ms.jzam

Amazing the VPN started to work again when I deselected Strict IP Address Check.

Re: Why Did Strict IP Address Check Break this VPN?

ms.jzam — Fri, 02 Feb 2018 15:35:23 GMT

Ok that's two confirmations on fixing the issue. I think this deserves to be bumped until we can sniff out a solid understanding of what's happening here.

BUMP!

Re: Why Did Strict IP Address Check Break this VPN?

BPry — Fri, 02 Feb 2018 15:59:22 GMT

@ms.jzam,

I would love to help on this but unfortenately I can't reproduce the issue at all. Unfortanetly the only way you can enable Packet Drop Logging is if your device is in Common Criteria (CCEAL4 Mode), which I doubt yours are; that would be something to check out though, because if they are you might get your why answer.

Re: Why Did Strict IP Address Check Break this VPN?

ms.jzam — Fri, 02 Feb 2018 16:05:06 GMT

@BPry

Happy to provide more detailed configuration for any attempts at duplicating the issue. What would be needed?

Re: Why Did Strict IP Address Check Break this VPN?

BPry — Fri, 02 Feb 2018 16:10:06 GMT

The exact ZP settings that you actually had selected at the time you ran into the issue; along with how you actually have the tunnel configured and the IP ranges being used on both sides. Then it would just be how your VPN was actually setup and configured. If you feel more comfortable sending this directly and not posting it on the forum just let me know and you can just email it over to me.

Re: Why Did Strict IP Address Check Break this VPN?

ms.jzam — Fri, 02 Feb 2018 16:53:22 GMT

@BPry I tried to find a DM feature, don't think there is one. Happy to continue over email.