Prove the firewall innocent?

[rjdahav163]

Hello,

We have communication from load balancer to 2 servers called S1 and S2.

Both S1 and S2 have same Windows Server version and have exact same patch level.

Both serve a web-service hosted on their IIS.

On Palo firewall, we have App-ID based rule allowing web-browsing as application.

Till 2 days ago, both of them were working perfectly. Yesterday, connections to S1 started being dropped since the App-ID got identified as something else on the firewall. Since then it has been like that. S2 is still working with App being identified as web-browsing on the firewall.

There were no App-ID updates on the firewall. The server team claims nothing changed on both S1 and S2.

We were engaged in long call with PA tech support and they did packet capture and still were not able to find anything. Palo asked us to check on server side since they think something must have changed on the server.

So it has now turned into a blame game between the server team and the firewall team.

Any idea how we can prove firewalls to be innocent?

OR

It is really the firewalls at fault and so has anyone experienced such an issue?

[BPry]

@rjdahav163,

Fundamentally if the firewall is seeing a different App-ID than allowed and you're dropping the traffic to S1, that's a firewall issue. Once you verify the traffic I'd expect to at least temporarily allow the new identified application through to S1 regardless of why the firewall is identifying it improperly. 

There's really only two reasons that the firewall would start suddenly identifying traffic differently, and that's signature updates that could potentially cause a false-positive detection on your existing traffic flows or the traffic itself changing. Since you don't mention what the traffic is being identified as, it's hard to help sort that issue out. It could have easily been a signature update that just happens to be identifying this traffic improperly, or it could be that the traffic changed. 

What app-id is the firewall identifying the traffic as currently? 

[Mike_Canfield]

interesting.  following this thread, as the date and description of your issue is too similar to an issue that started for us yesterday.  

We have two types of users ( call them laptop and VDI).  Both types use a phone agent from NICE called Max for contact center calls.  Monday at CoB no issues.  Tuesday morning issues, but only for VDI users.  Traffic for both users pass the same firewall rule.  But for an unknown reason traffic from VDI users is now being classified as quic-base.  since this was not allowed by FW policy, it was dropped.  The support team for VDI swears that no updates were made for either the VDI images, or for the VDI server itself.

Sure fundementally its the firewalls issue, but since it was working and now is broken for only a subset of users, specifically who use VDI.