NO duplicate addresses within address-groups permitted after upgrade to Panorama 10.0.7 from 9.1.8. Expedition created firewalls were affected

[tayloa29]

We upgraded our Panorama from 9.1.8 to 10.0.7 over the weekend.  As we attempted to commit the upgrade to the Panorama, we encountered the new requirement that Panorama 10.0.x code enforces no duplicate address objects allowed within an address group.  We have over 7700 address groups defined and this was a 5 hour exercise to run the commit many, many times to find the problem address groups, remove the duplicate objects within them and then commit those changes to Panorama and run the commit again, in order to find the next one.  We have built a number of the firewalls on this Panorama (converting Cisco ASA configs to PA VM-500s).  We are running Expedition version 1.1.65, Spark Dependencies 0.1.3-h2 and Best Practices 3.21.3.   We always run the remove duplicate objects step(s) as we prep a configuration.  We then merge the addresses and address groups into our Panorama and finally merge the security policies to the appropriate device group to get the rules setup for the new firewall. 

Is it possible that Expedition allows duplicate addresses to remain within an address group if converting from an ASA config?  Is this something that has been allowed with Expedition and if so, is there a later version that would either call this out to fix, or possibly auto-correct this problem for a specific address group?   We know we'll need to check our processes and see if any of our merge CLI commands were introducing this problem within our Panorama while building configs on the 9.1.x code and earlier. 

Any thoughts, suggestions or feedback would be greatly appreciated.      

[swaschkut]

to avoid manual cleanup of your configuration, there is already an automation script available since years, to help you there:

PAN-OS-PHP
https://github.com/PaloAltoNetworks/pan-os-php

pan-os-php type=xml-issue in=api://MGMT-IP out=output.xml
of course the output.xml must be loaded back into device:
pan-os-php type=upload in=output.xml out=api://MGMT-IP loadafterupload