Expedition support for PanOS9.1.1

[FilipElsen]

Dears,

We've needed to upgrade our Panorama & firewalls for bug fixing reasons to PanOs9.1.1.

Since the Panorama upgrade our migrations using Expedition are having issues:

PanOS9.1.1 with Expedition:

• Expedition pre 1.1.56: after a merge and config import, following is the config load error on Panorama: 

              "job failed because of configd restart"

• Expedition 1.1.60: config import works, but source and/or destinations objects are missing. This occurs repetively. (80% on security and NAT rules).

Is this known?

Which version of Expedition is supported for PanOs9.1.1?

Best regards,

Filip Elsen

[lychiang]

Hi Filip,

The latest version 1.1.61.2, it should work with PanOS9.1.1, please try update the expedition. 

[FilipElsen]

Hi Lychiang,

We'll give it a go and will let you know the result.

At the time of the last migration, the latest Expedition version was 1.1.60, which has issues with PanOs9.1.1.

Best regards,

Filip Elsen

[adevegamir]

Hi Filip,

Is possible to send us the config to tested this, please? Send to email fwmigrate@paloaltonetworks.com to study this case.

Thank you,

Regards,

[DamienDove]

Hi Filip, did you resolve this issue?

I'm loading a migrated config into 9.1.1 and the source and destination objects are missing.

[FilipElsen]

Hi @lychiang,

During previous COVID months, migrations were put on halt, but are restarting as of last week.

The firewalls and Panorama are on 9.1.1

We're on Expedition: 1.1.69.2 and still have issues.

The config had been prepared as before, but when importing it into Panorama we always hit: 

"job failed because of configd restart"

Best regards,

Filip

[FilipElsen]

Hi @DamienDove 

No - the issue is not solved.

When loading the migrated config, it works, but when performing a commit we get an error:

"job failed because of configd restart"

Best regards,

Filip

[FilipElsen]

Hi,

@sjanita @aestevez  @dgildelaig 

During the last COVID months, our migrations were put on hold and restarted last week.

The Firewalls & Panorama are on 9.1.1

In Expedition version:

• 1.1.60, Loading the merged configuration in Panorama worked. Committing the configuration in Panorama worked. Only this version had an issue since it was removing the source-ips, destination-ips and ports while merging the config. Similar like @DamienDove.
• 1.1.69.2 In this version, the config merge etc is working. Loading the merged config into Panorama following error is received:

Can you please have a look, we're blocked for the moment.

Thanks a lot,

Filip Elsen

[dgildelaig]

Just to discard options, there was nobody performing a management service restart during your commit process, right?

[FilipElsen]

Hi Didac,

Right!

We don't do this command often, which is good 🙂

Best regards,

Filip

[lychiang]

Hello @FilipElsen ,

Instead of load full config, can we try do do "load cofig partial" command from Panorama CLI to see you encounter the same issue? Also what platform is your panorama? 

[FilipElsen]

 Hi @lychiang, @dgildelaig 

We have an M500 cluster with full disk extension.

I was never a big fan of the "partial" and it showed off again today.

While loading the partial config, the primary Panorama became unresponsive and I needed to failover & revert.

Such symptoms I encountered in the past, therefore I avoid to do it like this command.

It has been going fine for +1Y and we've performed about 21migrations (Checkpoint Central policy sent to 2 different datacenter clusters, which brings quitte some complexity (routing, nat, auto-zone assign etc). (Spoke @dgildelaig about this during an event).

I've provided the export of the project, the merged xml.

Can you please shine your wisdom on it?

Best regards,

Filip

[lychiang]

Hi @FilipElsen 

This is not a Expedition issue , it has been identified as issue in PAN-OS 9.1.1 , please review below address issue in PAN-OS 9.1.2

PAN-133378

Fixed an issue in Panorama where a process ( configd ) restarted while doing a commit using a RADIUS super admin role.

https://docs.paloaltonetworks.com/pan-os/9-1/pan-os-release-notes/pan-os-9-1-addressed-issues/pan-os...

Thank you!

[FilipElsen]

Hi @lychiang @dgildelaig @DamienDove ,

Our Panorama's have been upgraded from 9.1.1 to 9.1.2.

The merged config file has been imported, it takes a while!

2020/06/04 16:08:13   16:08:13       186939                                  BuildXMLCache                            ACT   PEND        47%

2020/06/04 16:07:26   16:07:26       186938                                  BuildXMLCache                            ACT   PEND        69%

2020/06/04 16:06:48   16:06:48       186937                                  BuildXMLCache                            ACT   PEND        93%

2020/06/04 16:06:13   16:06:13       186936                                  BuildXMLCache                            ACT   PEND        99%

2020/06/04 15:29:33   15:29:33       186935                                           Load                            ACT   PEND        99%

In total it took about 35 minutes that the process was on Load / PEND at 99%.

In the end, the import worked.

Thanks a lot for the solution!

Best regards,

Filip Elsen

[FilipElsen]

Hi @lychiang @dgildelaig 

With PanOS 9.1.2 the import itself works, the configd restart error is solved. Nevertheless it takes a lot of time.

When reviewing the policy, we noticed that again the source/destination/service objects got lost, even though the are found into the XML.

Example 1

Expedition

Panorama

Example2:

Expedition

Panorama

I've made a search in the XML for "rule 229" above, but this seems to hold the correct values.

Seems like compatibility is lagging with PanOs 9.1.X.

We're blocked on our migrations.

Can you please shine a light?

Thanks a lot,

Filip Elsen