PAN-OS 10.1.9 -- CAUTION

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Announcements
Please sign in to see details of an important advisory in our Customer Advisories area.
Palo Alto Networks Approved
Palo Alto Networks Approved
Community Expert Verified
Community Expert Verified

PAN-OS 10.1.9 -- CAUTION

L6 Presenter

I would highly recommend waiting on deploying 10.1.9 in your enviornment, whether it be large enterprise or SOHO.

 

Recently my company had upgraded a 5250 pair from 10.1.8 to 10.1.9 as we had experienced 2 service impacting bugs.

 

After upgrading 10.1.9 we found that the FW was exhibiting odd behavior where traffic which had specific allow rules for it wasn't matching the defined allow rules and was instead being denied all together or was matching a different rule lower in the policy stack.

 

What made troubleshooting this further was the traffic which was being denied, that shouldn't have been, could only be seen in the session browser as being denied.  Even hours later the incorrectly denied traffic was nowhere in the GUI logs; and could only be found in the CLI by using the session ID from the temporary session browser view while the session existed there.

 

We had a log P1 TAC case that unfortunately found no cause for this behavior so we downgraded back to 10.1.8 and all services were restored.

 

Traffic types effected were:

 

IKE

IPSec

GRE

Non-decrypted HTTPS TLS1.2 (That we saw)

1 accepted solution

Accepted Solutions

L6 Presenter

This case went on forever unfrotunately, but we ultimately found one of the causes of our issues.  10.1.9 changed the behavior of how the firewall responded to an ARP.

 

Prior to 10.1.9 the firewall would respond back out an egress (outgoing) interface different from the ingress interface.  Which meant that a NAT policy could have an interface the that traffic didn't originate from.  In 10.1.9 Palo changed this behavior where the FW would (only send a pack back egress that it ingressed) match a NAT policy where the incoming and outgoing interfaces were the same.

 

Palo by changing this behavior ultimately made our NAT policy no longer work.  We updated our NAT policy to match an "ANY" outgoing interface instead of a specific one and doing so solved our upgrade issue with traffic failing after upgraded:

 

PAN-195114 -- though in our instance we have a single VR not 2 as the bug-ID mentions.
Brandon_Wertz_0-1701100508033.png

 

IMO -- This change in behavior shouldn't have been a "patch" and should have come in a major PAN-OS upgrade, not a minor release.  But Palo saw this as a "bug" hence they released it as a patch/minor release update.

View solution in original post

9 REPLIES 9

Community Team Member

Hi @Brandon_Wertz ,

 

Thanks for sharing! This is great feedback that can be helpful for our users. 

LIVEcommunity team member
Stay Secure,
Jay
Don't forget to Like items if a post is helpful to you!

Please help out other users and “Accept as Solution” if a post helps solve your problem !

Read more about how and why to accept solutions.

Cyber Elite
Cyber Elite

10.1.9-h1 was just released to the public, however it doesn't look like the release notes have been published just yet. 

Cyber Elite
Cyber Elite

OK so 10.1.8 is pretty good?

Cyber Elite
Cyber Elite

To piggyback on the topic.

March 7 dynamic update updated threat id 32968 that has loads of false positives in SMTP traffic and no fix out yet...

Enterprise Architect, Security @ Cloud Carib Ltd
Palo Alto Networks certified from 2011

L0 Member

We are having the same problem with IKE and IPsec. After downgrading to 10.1.8 everything is working again.

L3 Networker

Interesting, we're seeing issues after upgrading to PAN-OS 10.1.9 as well, coming from 10.1.8-h2.

Both internal and external traffic has started to act weird, causing reconnects to on-premise Skype for Business as well as issues with an external cloud call center solution. I was unable to find anything in the known issues list that would explain this, but I'll try to downgrade back to 10.1.8-h2 to see if it is resolved there.

L6 Presenter

This case went on forever unfrotunately, but we ultimately found one of the causes of our issues.  10.1.9 changed the behavior of how the firewall responded to an ARP.

 

Prior to 10.1.9 the firewall would respond back out an egress (outgoing) interface different from the ingress interface.  Which meant that a NAT policy could have an interface the that traffic didn't originate from.  In 10.1.9 Palo changed this behavior where the FW would (only send a pack back egress that it ingressed) match a NAT policy where the incoming and outgoing interfaces were the same.

 

Palo by changing this behavior ultimately made our NAT policy no longer work.  We updated our NAT policy to match an "ANY" outgoing interface instead of a specific one and doing so solved our upgrade issue with traffic failing after upgraded:

 

PAN-195114 -- though in our instance we have a single VR not 2 as the bug-ID mentions.
Brandon_Wertz_0-1701100508033.png

 

IMO -- This change in behavior shouldn't have been a "patch" and should have come in a major PAN-OS upgrade, not a minor release.  But Palo saw this as a "bug" hence they released it as a patch/minor release update.

Cyber Elite
Cyber Elite

So assuming your WAN interface runs on ethernet1/1 your DNAT policies (traffic from Internet to inside) had random destination interfaces attached?

 

Something like this?
Source zone - WAN
Destination zone - WAN

Destination interface - ethernet1/3

Enterprise Architect, Security @ Cloud Carib Ltd
Palo Alto Networks certified from 2011

Kind of, but not really.

 

The external side of our FW is all L2.  We have multiple L2/L3 segments on our WAN/INet beyond the FW.  There are 2 Border routers beyond the firewall own the L2/L3, which the firewall sees as the WAN.  The BRs are EIGRP neighbors with HSRP for the multiple L2/L3.

So in the NAT policy we just "picked" a "random" L2 network as the egress interface, because for us it really doesn't matter if it's the "right" or matching ingress interface because the BRs own all of them.

 

So the BRs have:

WAN-1 (WAN-Zone)

WAN-2 (WAN-Zone)

WAN-3 (WAN-Zone)

 

The FW participates in WAN-1, WAN-2, WAN-3 from a L2 perspective.  The ingress might have come in on WAN-1, but the NAT might have said egress = WAN-2, and it worked because the next hop was the BR which has/knows about all those L2 segments.  

 

But with palo "fixing the bug" the egress NAT interface needed to match the specific interface even though they are all a part of the same zone..

  • 1 accepted solution
  • 2624 Views
  • 9 replies
  • 4 Likes
Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!

The LIVEcommunity thanks you for your participation!