Action and Session End Reason conflict when SSL decryption enabled

[nikoo]

Hi,

SSL decryption was turned on for one of the inside servers. Although it looks good, but some of the logs are rather strange.

There are sessions like these:

Basically  Action - Allow, Rule is hitting correct one (the one permitting the traffic), but Type is Deny and Session End Reason is policy-deny. That looks false to me and it seems that traffic is permitted indeed. Can check more with captures, etc., but has anyone seen such an effect?

 PAN-OS: 8.0.2

[Brandon_Wertz]

I could be wrong, but to me it looks like this is normal.  This is why I think so:

You're getting an "allow" because it's matching a layer 3 / 4 "allow" but then if you were to look at the magnifying glass you'd see a deny type log (for Layer 7 controls) but the details of those specific logs would help clarify more as to what exactly was going on.

[BPry]

@nikoo,

Does this eventually shift away from SSL and into a more specifc app-id. One thing to keep in mind as well is that your traffic could be attempting to shift to 'web-browsing' and if you have your service set as 'application default' obviously web-browsing no-longer matches your rule. Without knowing the detailed logs or what your policy actually looks like I wouldn't be able to say for certain. 

[Remo]

Today we also found a lot of these entries on our firewall. The strange thing is: its exacly this log entry as on the screenshot from @nikoo, no url log, no threat log ... nothing. Only this type deny log with the action allow and the session end reason policy deny

Edit: I have now seen these logs on 3 different hardware series running 8.0.2. So the only thing I know, it is not hardware related

[BPry]

@Remo @nikoo,

Could you guys post your dynamic update versions so we can see if any of those match. It sounds like you'll both likely want to open a Tac case or reach out to your SE to pass along the infromation, but if we could find any additional common ground that may be shared other than simply 8.0.2 that would be great. I imagine that while 8.0.2 is a shared trait if it was specific to that software version by itself the issue would have already presented itself. 

[Brandon_Wertz]

I'm running 5060s on 7.1.8:

 
Application Version 709-4078 (06/13/17)

Threat Version 709-4078 (06/13/17)

Antivirus Version 2279-2767 (06/19/17)

I saw the same type logs on my FW.  I think the this might be "normal" as described by my comment above.  Maybe not though, unless some can refute my assumptions?

[Remo]

2 cluster have the same versions mentionned by @Brandon_Wertz installed and 1 cluster has app&threat 708-4066 and av version 2278-2766 installed.

@Brandon_Wertz

Of coulse it could be absolutely normal ... but when I read the first post on this topic and then checked my logs I thought there should be another log (url/threat) which indicates the reason of this "policy deny"

[Brandon_Wertz]

Here's a detailed log view from one session on my FW

[Remo]

@Brandon_Wertz

Thats how I was expecting the log should look like. But at least on 8.0.2 other logs exept the traffictype:deny-action:allow are missing.

Edit: One cluster was updated on saturday from 7.0.x to 8.0.2. Since then it started with these missing logs. On 7.0.x I haven't had such logs at all with these type/action/session-end-reason --> So it could be a bug of 8.0.2

[nikoo]

Had to run home yesterday, so didn't elaborate much on details. 🙂 But, yea, there are no additional inspection events related to that specific event - just a one rule hit, which covers all of the possible variations for application shifts on that TCP/443 port, without using application-default behavior. These logs definitely showed up only after enabling SSL decryption for that specific inside server and dissapeared after disabling it, so that may be related to some application shifts happing somewhere under the hood or some inspection profile having some impact.

Here's a sanitized full log entry:

And here's a rule:

There are some inspection profiles attached to it, so there is an option to disable them and see if that has any impact on the logs. At the moment decryption is disabled though, so cannot test that.

8.0.3 is out - I don't see anything related in the fixed bug section though, but may worth trying.

Cannot tell about 7.0.x or 7.1.x, because had to upgrade to 8.0.x in order to have Inbound SSL decryption for ECDHE.

Sitting on 708-4066.

[Remo]

I've opened a TAC case to find out what's going wrong here. Will get back with an update as soon I have new informations.

[nikoo]

@Remo Hi, did you get any feedback from TAC?

[Remo]

hi @nikoo

Unfortunately no useful result so far ...

[Remo]

FYI: These logentries are ALL because of decryption errors. In PAN-OS 8 mostly because of client certificates as almost anything else should be decrypted by default. Support is now verifying if it is "expected behaviour" that this "special" decryption-error is shown as "policy-deny" or if this a bug and the log is expected to show "decryption-error".

[Remo]

FYI: "The issue is due to a current limitation in identifying session end reasons with SSL code values, which is expected to be fixed in the upcoming maintenance releases (ETA unknown). As of now, the session-end-reason is working as designed and uses the generic "policy-deny" for certain failure condition."