OSPF and Cisco Routers

[jsalmans]

Greetings all,

I was doing some Core routing work during an outage this last week and ran into a repeat of some issues we had when we initially put our PAN boxes in to place.  The original scenario:

• A subinterface existed on the Palo Alto with the tagging set for a point-to-point vlan
• That vlan connected the Palo Alto directly to a Cisco 4500-X VRF.  VRF was used for traffic separation
• Our original Core vlan also connected to this VRF and was then sent out to each core router (6500-E series with Sup720).  Each core router had an SVI on this vlan making it logically Layer 2 to the 4500-X.
• Jumbo frames were enabled and mtu set to 9150 on the Palo Alto subinterface, the relevant port channels, and the SVIs on the 4500-X and all of the core routers.
• Policy Based Routing is applied on other SVIs to push the traffic over to the core routed vlan and then up to the firewall

With that original scenario, we initially had issues coming online with OSPF.  At the time, I believe we thought it was an issue with the 4500-X and Cisco TAC recommended adding an mtu ignore command on the core routers which brought everything online.

Fast forward to this last week.  Cisco has advised us to remove as much Layer 3 from the 4500-X as possible leaving it to just be a Layer 2 10G aggregate which is what it is good at.  New design:

• New subinterface on the firewalls with tagging to match the vlan the core previously used to link to the 4500-X
• Removing VRF from the equation on the 4500-X so it just passes the vlan from the core routers to the firewalls through Layer 2
• PBR is still set on SVIs in the core to push traffic to the firewall

First issue I ran in to was that a dead-timer was set on the core devices to around 600 seconds.  Palo Alto doesn't have a direct dead timer setting and I think we would have had to specifiy hello-timers on the core devices to fiddle with the numbers enough to make the math come to 600 seconds on the Palo Alto... we ended up removing the dead-timers for now.

Second issue was mtu again.  The mtu ignore was still set on the Cisco routers so I'm confused why it was an issue but one core router was stuck in exstart and the rest showed connected through OSPF but it seemed like the routes weren't shared.  I set the subinterface on the firewalls back to 1500 and then set it on the first core router and we pretty much instantly had connectivity and routes.  I proceeded to set it 1500 on the rest of the core to get us back online for the evening but I'm concerned about leaving it there.

I wanted to ask here and see if anyone else has had any sort of difficulty getting anything other than 1500 mtu and default OSPF options set while trying to form a link between Palo Alto and Cisco devices?

Thanks!

[9t89m8fu]

Cisco TAC recommended adding an mtu ignore command on the core routers which brought everything online.

This was bad advice. Tricking the devices into thinking their MTUs match can result in a DBD packet being sent that is too large for the recipient to process, leaving you stuck in exstart, as you have seen. You need to fix the MTU mismatch, not hide it.

[BPry]

@jsalmans,

As @9t89m8fu mentioned I would look at fixing the MTU mismatch as soon as possible, Cisco TAC should have never told you to use the mtu ignore command. What you are trying to do is going to require an outage with enough time to work through all the issues, and I would attempt to get Palo TAC and Cisco TAC on the same line if they are willing. While you can plan for this without much issue and you'll design will look fine, in practice it usually requires quite a bit of special configuration on both ends to get everything playing nice together.