Firewall dropping HTTP only from specific source network

[mhill99]

Hello. I've come upon an extremely strange situation that I'm hoping to get some assistance on. I've already opened a case with Palo support, but they seem to be at a loss as well.

For one specific internal network, the edge Palo Alto is dropping HTTP (80) packets at the internal interface. Even more strange, it seems that packets are somehow changing to port 25 and the proceeding through proper packet filtering. But, because the destination (Internet) sites aren't hosting SMTP services, of course that traffic is being reset.

Troubleshooting wise, I've enabled packet capture/filtering and ran the command "show counter global filter packet-filter yes delta yes severity drop" and the one recurring error found is:  Packets dropped: forwarded to different zone

Unfortunately, no changes have been made recently to the routing table. All sessions have been cleared (restarted the firewall). I've been deleted some legacy zones to be sure they weren't causing issues. I've also tried enabling a zone protection profile and allowing asymmetric routing, but that didn't help either.

Using "test routing fib-lookup virtual-router "default router" ip <destIP>" shows that proper routing will be used. Using the same command and testing the return traffic routing also shows proper hops.

Last couple notes, if I change the routing to circumvent the firewall completely (using a separate ISP), HTTP traffic works properly. When going through the firewall, all other traffic works just fine. PING, Traceroutes, SSL, etc so I don't think it's anything to do with routing though the error seems to point that way. 

Any help would be much appreciated.

[BPry]

@mhill99,

That's an extremely odd thing to run into. Out of curiosity, have you verified that you don't have a specific NAT statement or PBF rule in place for this specific source network? That is the only thing that I can think of that could potentially be changing the source port like that if routing is working properly when you take the firewall out of the equation. 

[Adrian_Jensen]

Adding on to what BPry said, do you have a NAT Policy rule with a Translated Packet option "Translated Port" set to port 25? I.e. you created a policy to NAT traffic to/from your mail server and explicitly set the translated port option (perhaps not setting the Original Packet service), instead of allowing the original destination port to take precedence. 

[mhill99]

Thanks, guys. Unfortunately, there is no PBF policy whatsoever and I've verified that outbound traffic is hitting the correct NAT policy for overload. No static translated port translation.