traffic disruption on routing change

[COsterbrink-LIS]

hi folks,

I deleted a vlan subinterface from the VR config and from the ospf redist options. after comitting the change we had a partial service disruption where sessions where dropped. I can see a visible drop in thoughput but not in number of sessions. according to monitoring traffic I can says that it lasted roughly 15 seconds which is quite long. I can see in the system log that the route daemon was restarted but that seems to happen with every commit and usually case zero impact. but in this case the routing config was changed. I can't see any ospf messages in the system log and also not on the router behind the fw. so the traffic disruption seems just to be internal on the fw. has anyone experienced the same? I would be interested if this is expected behavior? 

BR

Chris

[BPry]

@COsterbrink-LIS,

Might help if you post your PAN-OS version that you were running at the time. You'll have more detailed OSPF event logging within routed.log, but that file rotates pretty quickly so unless you pulled it at the time of the disruption you'll want to probably look at routed.log.old and see if it still has the logs for the time period of the issue. 

[COsterbrink-LIS]

Hi  @BPry 

thanks for your reply. PANOS is 10.2.16-h6 - unfortunately the logs in  routed.log.old also rotated out. 

but we have planned further migrations of vlan sub-interfaces which need then to be deleted and I will have a close look to the routed.log and also on the upstream ospf router  

in system log there were no suspicious messages. will keep this post updated. 

[COsterbrink-LIS]

just to add the newest insights, we got really bad trouble when deploying any templates but also sometimes on device-group pushes. From the routed.log you could see that each push the ospf adjacency to all ospf neighbours was teared down by the Palo and had to be reestablished which took some seconds to complete. we disabled the strict lsa check feature on the Palo (wasn't enabled on the Cisco peer routers)  but to our understanding this should only affect the ospf process if the opposite peer is in helper mode and it restarts the ospf adjacency and things do not match (new lsa to old lsa). Also we found that one static route we exported via ospf didn't exist anymore and was now disributed by one of the cisco routers via ospf instead. We also removed that explicit route redistribution and since then everything seems to run smooth. Maybe strict lsa checking in addition with propagating and receiving the same route via ospf caused the restarts... it's only speculation, but now it runs as expected.