We have been trying to troubleshoot an issue for a couple weeks now. We seem to have found the issue, but it doesn't make sense. I'm hoping someone can shed some light on this:
First off, we have a pair of PAN-4020's in HA @ 3.1.9.
I'm going to oversimplify the situation, but I think the logic will hold:
Segments A, B and C
Segment A is a user segment with 2 routers, in serial, (layer 3 switches) between the test user and the PAN
Segment B is our connection to the Internet via a Cisco router connected to the PAN
Segment C is a Data Center segment with one router (layer 3 switch) between the servers and the PAN.
It's all layer 3, so this doesn't matter much, but Segments A, B and C hit the PAN on different interfaces.
Segment C also has a load balancer on it (same device as the layer 3 switch).
Users on Segment A are dynamically NAT'd ("many" users to 1 public address)
Servers on Segment C are 1:1 static NAT'd
Load balancer uses public addresses; All users access the servers via the public address, whether local or from the Internet.
From a workstation on Segment A, if I web/ssl to a server on Segment C it works fine (there is a policy allowing this)
From a workstation on Segment A I can go out on the Internet (obviously also a policy)
From a workstation on Segment A, if I try to traceroute to a server (public or private address) on Segment C it fails. This is where the confusion lies ..
There is not a policy allowing ping/icmp from A to C. What I don't understand is I do NOT see any failures in the logs from the workstation on A (either its public or private address) to the server on C (also checking both public and private addresses). ALSO, the traceroute goes off to the Internet (Segment B) and eventually fails rather than heading toward Segment C. Lastly, once I do put an explicit policy in place allowing icmp/ping (and SSH for that matter) from Segment A to Segment C, everything is happy. I don't understand how a security policy is impacting routing paths.
Let me review what I understand
Segment A is your trusted zone
Segment B is your untrusted zone
Segment C is your server zone.
For traffic to pass between these zones you have to configure security and NAT policies.
For traffic to get from segment A (internal user) to segment B (website on someonet else's network) you have to create a security policy that allows that traffic and a NAT policy to translate the source traffic to the assigned external IP.
For traffic to get from segment B (Internet user) to a load balanced server on segment C you have to create a security policy for the allowed traffic and a No translation NAT policy for the source and destination
For traffic to get from segment A (internal user) to a load balanced server on segment C you have to create a security policy for the allowed traffic and the appropriate translation of the source traffic.
Your ping/traceroute traffic was not going off into segment B, it did not have a policy to allow it to the proper destination and was following the default route on the PaloAlto on which it could go.
Additionally if Segment A and Segment C are in the same zone, there would need to be an intra-zone rule that would allow it. (Source and destination zones are both the trust zone)
I need to parse most of what you said to fully understand it, but I don't think my original questions are answered:
1) why is traffic following the fw's default route rather than the known route of the destination server? Granted, without a security rule in place, it isn't going to work, but it should still try .. and that leads to #2
2) why do I not see "deny"'s in the monitor log?
It really appears that the fw is making an assumption rather than routing. It makes no sense to me that a security policy impacts the routing path.
"Your ping/traceroute traffic was not going off into segment B, it did not have a policy to allow it to the proper destination and was following the default route on the PaloAlto on which it could go."
I think this is where I am confused. How do you define a "policy" that controls routing? Shouldn't that be an entry in the Virtual Router/Static Routes?
Forget about routing, the PAN is more than a layer 3 device.
Routing will not occur if there is not a rule. There is an implicit deny in place at the end of the policy rules. The implicit deny does not log, if you want to see what is being denied you will need to create an implicit rule.
Can you provide an output of what a ping or traceroute shows when the rule allowing ping is not in place or disabled.
Click Accept as Solution to acknowledge that the answer to your question has been provided.
The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!
These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!
The LIVEcommunity thanks you for your participation!