topic Re: BGP Multiple ISP VR Requirements in General Topics

BGP Multiple ISP VR Requirements

KyleFreise — Wed, 06 Jul 2016 19:44:00 GMT

I'm attempting to wrap my head around a very critical piece of setting up BGP between 2 ISP's concerning how many Virtual Routers are required.

I currently have 1 ISP (A) up and running on BGP just fine and my other ISP (B) will be converted to BGP on Monday. Both will be advertising my public IP space from ARIN.

So my question is, do I put both ISP A's and ISP B's interfaces on the same VR or do they need to be on their own separate VR's?

I'm receiving a few conflicted answers to this question, so I'm looking for real world experience.

Currently I'm still a strong believer in that I need to just add my ISP B as its own Peer Group to my current primary Virtual Router, add ISP B's own subinterface to the Resdistribution Profile along with ISP A's subinterface, configure a new Import Rule for ISP B plus an Export Rule with a Prepend of 2 if I choose to, and finally check ECMP for "load balancing".

All this sounded great until I was told the following from a Palo rep: "Two interfaces ( belonging to same VR ) cannot have IP addresses from the same subnet. x.x.x.x/24"

That threw a wrench into my whole thought process because both ISP's are advertising my /24 from ARIN, but are physically connected with 2 different /30's to each ISP's switch.

Re: BGP Multiple ISP VR Requirements

pankaku — Thu, 07 Jul 2016 16:18:56 GMT

You can have one VR. Prefer one BGP peer and if that BGP peer goes down then it will automatically start using second ISP

Check this KB

https://live.paloaltonetworks.com/t5/Configuration-Articles/How-to-Prefer-a-BGP-Peer-for-Installing-a-Received-Prefix-in-the/ta-p/60071

Re: BGP Multiple ISP VR Requirements

KyleFreise — Thu, 07 Jul 2016 16:46:27 GMT

My import rules are default routes (0.0.0.0/0) so wouldn't ECMP be a better model for load balancing?

I do agree now though that a single VR is all that is required.

Re: BGP Multiple ISP VR Requirements

pulukas — Sun, 10 Jul 2016 10:43:42 GMT

If you want to load balance then your options are to use ECMP (if you have PanOS 7 where this feature was introduced) or configure policy based routing. With policy based routing you choose what types of traffic by source or other criteria to forward to which ISP.

ECMP

ECMP Load-Balancing Algorithms

Configure ECMP on a Virtual Router

Enable ECMP for Multiple BGP Autonomous Systems

Policy Based Routing

Re: BGP Multiple ISP VR Requirements

KyleFreise — Tue, 26 Jul 2016 13:16:14 GMT

I tied each ISP's BGP link into one VR and what I was originally confused about was how to tie our /24 ARIN space in with these links.

In short I didn't realize you can apply the /24 public space to any interface and just advertise it out with the Redistribution Profile. I was thinking it had to be applied to both external ethernet ports.

Not the case, just tie the /24 to one of the external facing ports as subinterfaces and then I put a simple /32 on the other ISP's subinterface for ping testing. Not really necessary, but I was playing around.

Re: BGP Multiple ISP VR Requirements

MatthewRaguso — Wed, 02 Nov 2016 18:38:28 GMT

@KyleFreise wrote:
I tied each ISP's BGP link into one VR and what I was originally confused about was how to tie our /24 ARIN space in with these links.

In short I didn't realize you can apply the /24 public space to any interface and just advertise it out with the Redistribution Profile. I was thinking it had to be applied to both external ethernet ports.

Not the case, just tie the /24 to one of the external facing ports as subinterfaces and then I put a simple /32 on the other ISP's subinterface for ping testing. Not really necessary, but I was playing around.

What happens if the physical interface goes down? Will the sub-interface go down as well?

Re: BGP Multiple ISP VR Requirements

KyleFreise — Wed, 02 Nov 2016 19:25:19 GMT

Yes if the primary interface (say ethernet1/18) goes down, then the subinterface (ethernet1/18.1) will go down as well.

I've yet had that happen on the BGP links as my issues stem from downstream routing issues during maintenance or routing equipment failure down one of the pipes.

Re: BGP Multiple ISP VR Requirements

MatthewRaguso — Thu, 03 Nov 2016 14:21:00 GMT

This is currently the approach I am using. I was just wondering if there was a better way. Thanks for the reply.

Re: BGP Multiple ISP VR Requirements

RFalconer — Fri, 04 Nov 2016 17:35:18 GMT

Can you explain a little more how you use a redistribution profile to advertise prefixes with BGP to your ISP? I have BGP running on many PAs and just use standard BGP with export rules and redistribution rules, no subinterfaces in use.

Re: BGP Multiple ISP VR Requirements

KyleFreise — Fri, 04 Nov 2016 19:29:45 GMT

My redistribution profile is using Source Type "connect" and the interfaces are the main untrust facing interfaces as well as their respective sub-interfaces.

The import rule is simply matching and allowing 0.0.0.0/0, while the export rule is matching the ARIN space we have. Then the redistribution profile is referenced within the 'Redist Rules' tab for the BGP configuration.

Again, I'm only redistributing outward facing interfaces (including the subs).

Re: BGP Multiple ISP VR Requirements

RFalconer — Fri, 04 Nov 2016 20:30:02 GMT

In the Redist Rules, did you ever try with just entering your public space subnet instead of referencing the redistribution profile?

As long as there is an entry for the public space in the routing table of the PA, this should work fine. Having a subinterface on the ARIN assigned network will satisfy this or you can create a dummy route, sort of like routing a prefix to null0 on a router.

Re: BGP Multiple ISP VR Requirements

KyleFreise — Fri, 04 Nov 2016 20:50:37 GMT

Can you add subnets to a Redist Rule directly? All I see are settings for Community, MED, Preference, etc.

Re: BGP Multiple ISP VR Requirements

RFalconer — Fri, 04 Nov 2016 21:13:40 GMT

Yep, just type the subnet directly in the box.

Re: BGP Multiple ISP VR Requirements

MatthewRaguso — Mon, 07 Nov 2016 21:56:35 GMT

That's interesting, I never took the time to read that line of text.

What are you using for your dummy static route?

Re: BGP Multiple ISP VR Requirements

KyleFreise — Mon, 07 Nov 2016 22:03:13 GMT

Dummy static route?

Re: BGP Multiple ISP VR Requirements

MatthewRaguso — Mon, 07 Nov 2016 22:06:47 GMT

@RFalconer wrote:
In the Redist Rules, did you ever try with just entering your public space subnet instead of referencing the redistribution profile?
As long as there is an entry for the public space in the routing table of the PA, this should work fine. Having a subinterface on the ARIN assigned network will satisfy this or you can create a dummy route, sort of like routing a prefix to null0 on a router.

Was just wondering how you would set that up on the firewall?

Re: BGP Multiple ISP VR Requirements

RFalconer — Thu, 10 Nov 2016 17:47:20 GMT

Here's an example.

The direct connection to the isp is a /30, 10.1.1.2/30.

Your public IP space is 128.1.1.0/24 (This doesn't have to be BGP advertised. It can just be another subnet that the ISP has assigned to you that they route to 10.1.1.2)

Create a static route for 128.1.1.0/24. I usually set the interface to the untrust and the next hop to 'None'

By creating this static route, it puts the prefix in the routing table. BGP will only advertise prefixes that exist in the routing table. Also, when NAT statements are created for the public subnet, an entry for this public network must exist in the routing table or the processing of the traffic will fail.

Check out page 4 on this doc:

https://live.paloaltonetworks.com/twzvq79624/attachments/twzvq79624/learning_tkb/189/2/DOC-1628.pdf

A forwarding lookup is done before NAT is even processed. If the destination doesn't exist, then traffic is discarded before it even gets to the NAT lookup. By having a route to null0, the prefix exists and the flow continues.

Re: BGP Multiple ISP VR Requirements

MatthewRaguso — Thu, 10 Nov 2016 21:44:31 GMT

From my testing, a static route isn't even needed when you specify the /24 subnet in the Redistribution tab instead of using a Redistribution Profile.

Thanks for the tip.

Re: BGP Multiple ISP VR Requirements

KyleFreise — Thu, 10 Nov 2016 21:51:31 GMT

That's correct.

Re: BGP Multiple ISP VR Requirements

RFalconer — Fri, 11 Nov 2016 23:16:19 GMT

Have you tested passing traffic into the subnet you're advertising, using NAT rules bound to the interface on a different subnet? I've never been able to do this without either a loopback or a null route.