Routing question - MPLS between two sites, with one of those connections being a failover ISP

[uscit]

Hello,

I've got a scenario in which I'm not sure how to proceed.

We have two sites, both sites just got new circuits.

In Site A, we have two new circuits (ISP #1 and ISP #2).  I've set up in my PA-500 Policy Based Forwarding to have ISP #1 as the primary internet connection, and if that drops, it will failover to ISP #2.  We've tested this and it works.

In Site B, we have one new circuit, also through ISP #2.  There is an MPLS private connection set up between Sites A and B through ISP #2, where PCs in Site B should be able to connect to server resources and files in Site A.

I have three interfaces configured on my PA-500:  Eth1/1 for the ISP#1, Eth1/2 for ISP#2, and Eth1/3 for LAN.  Because of the PBF failover rule, I have those three interfaces set up that way. I can ping from the Eth0/3 LAN interface on the PA-500 to a PC in Site B that is configured with a private LAN IP and the gateway pointing to the new circuit, so I know that the MPLS private connection is set up and the routing is correct.  However, I cannot be in the LAN zone with a PC on my side (gateway pointing to the PA-500 interface) and ping to that Site B PC.  I'm assuming that also means that the PC in Site B would only be able to ping up to the LAN interface on the PA-500, but not into the private LAN in Site A.  I've attached a diagram pic to give a visual.

I intially thought that I needed to add a static route for the private LAN subnet in Site B into the PA-500 in order for the connection to work, but I'm not sure how this works with the setup I have going (the ISP #2 being a secondary ISP line, but also where the private connection to the other site comes through).  In the "How to Configure Palo Alto Networks Firewalls when Connected to an MPLS Network" document, it talks about creating a separate interface for the MPLS connection, as well as separate zones/routes/policies.

What do I do here to allow connections through from Site A to Site B with the interfaces set up the way I have them?

[panos]

can you attach your virtual router config ? or try that

You can solve this by using 2 virtual routers for side A

eth interface of LAN for site1 use virtual router 1

virtual router 1 ----- default gw isp 1gw

virtual router 2 ----- default gw isp 2gw

also you need adding some routes

for vr1 -   for destination address "LAN of site2" subnet go to next VR and VR2

for vr2 -   for destination address "LAN of site 1" go to next VR and VR1

[uscit]

Current VR config (one VR):

Interfaces included are all 3 (Ethernet1/1, Ethernet1/2, Ethernet1/3)

One static route of:

Name               Destination     Interface     Type               Value                             AdminDistance     Metric     NoInstall

defaultroute     0.0.0.0/0          eth1/2          ip-address     <gateway of ISP #2>     default                    10

I hope that helps.  I was reading through the above suggestion of 2 VR's, and I'm wondering:  do I still need to do some sort of NAT/Security/Zones for the private connection from Site B to Site A?  It's still a cloudy thought to me on configuring access to Site A's LAN resources for Site B on a private connection with ISP #2, that will also be used as a secondary ISP for Site A in general.

[panos]

ok so you hava pbf rule for isp 1 is that right ?

can you write pbf rule(s) you have ?

[uscit]

Yes, one PBF Rule:

ISPFailover;  Source Zone is "LAN" on "Any" Address and User; Destination are all "Any"; action is Forward, egress through Eth1/1, next hop is <ISP#1 gateway>, enforce symmetric return is "false"; Monitor profile is "failover", target is 8.8.8.8, disable if unreachable is "false", schedule is "none".

[panos · Accepted solution]

So you have to add another PBF to top of this rule which has destination address as SideB LAN to route from ISP2 gateway

Because when you wrote any as dest. address it forwards evrything to ISP1

[uscit]

I created a second PBF as noted, destination of the Side B LAN, forward through Eth1/2 with next hop of <ISP#2 gateway>, and moved it above the failover PBF.

However, that doesn't help me as far as being in the PC in Site B and I can ping Eth1/2 (ISP#2 interface) but not Eth1/3 (LAN interface).

I added a security policy to allow any traffic from the ISP2 Zone to the LAN zone, but that didn't help either.

[panos]

you mean before writing that second pbf rule you can ping the interface but after that you can't

[uscit]

No, i couldn't before writing the 2nd PBF rule.

As stated in the original post, I can ping from the Eth1/3 "LAN" interface all the way to a PC in Site B, but if I configure my PC within the LAN to point to the PA as gateway, I cannot ping to the PC in Site B from there...so nothing actually in the LAN can ping out past Eth1/3.

Also, from the other end, I can ping from the PC in Site B all the way to the Eth1/2 "ISP#2" Interface, but I cannot ping from the PC in Site B to the Eth1/3 "LAN" interface.  I did a tracert for these pings from the Site B PC...when tracing route from PC to Eth1/2 "ISP#2", the whole trace completes.  When tracing route from PC to Eth1/3 "LAN", the trace hits the "Customer Serial" interface of the ISP router, and then times out after that.

Edited last sentence.

[uscit]

I'm back in office now, and was able to test a local PC on the LAN in Site A pinging to the PC in Site B after putting in the 2nd PBF rule, and it works.  Now, I have to figure out why I cannot ping through to the Site A LAN from the PC in Site B.  I have a feeling there is a missing static route on the Site A ISP's Router to the private LAN, since I can ping to the Eth1/2 "ISP#2" Interface but not to the Eth1/3 "LAN" interface.