Rest API and import

[hkp]

I need to add a lot of addresses (around 10,000) to my firewalls.  I have them in a list and have created a script to add them one at a time.  The problem is that this takes a long time.  I then tried to add them all, but my URI was too long.  So now I can add about 50 at a time.  It still takes a while.

I also know that with the import command I can import a full config.

My question is this: Is there a way with the import command that I can add several addresses at once?

Thanks.

[jvalentine]

Is there a reason that you need to do this with the API? If this is a one-time event, you could do it pretty quickly through the CLI using the "scripting-mode" function.  Here's what it looks like on a PA-4020 running 5.0.6 (via SSH console):

admin@pa4020> set cli scripting-mode on

admin@pa4020> configure

(now paste in the 10000 lines of objects)

admin@pa4020# set address object00001 ip-netmask 1.1.1.1

admin@pa4020# set address object00002 ip-netmask 1.1.1.2

...

...

admin@pa4020#

I also did the same thing on a PA-5050.  The first 3000 will go pretty quick, and after that it will slow down quite a bit.  The PA4020 was processing about 3-4 new objects/sec, while the PA5050 was adding about 7-8 entries/sec.  Should be do-able in ~20 minutes if I had to extrapolate. 

[bdeschut]

You could also export your configuration, add the objects manually in xml format and import again.

Shouldn't be hard to create a script to get the objects in PAN xml format:

for example:

             <entry name="device_name">

              <ip-netmask>10.1.1.1/32</ip-netmask>

            </entry>

Or you can load a partial configuration, but I have no experience with this yet.

How to Load Partial Configs PAN-OS 4.0

[hkp]

This is what I finally did.

Now that I have all my addresses in Panorama adding a single address at a time takes over a minute.

Is that expected behavior?

[mschuricht]

Please answer the following questions:

• How many objects do you have in total?
• How many device groups do you have?
• What is the RAM and CPU allocated to Panorama?

[bdeschut]

As mschuricht is implying, having a lot of objects in your configuration can cause an additional load / configuration latency on your device. When adding an address, the device will parse the entire configuration file to check if the objects already exists etc.

[hkp]

Fair enough.

I have over 50,000 objects and two device groups.

I just increased panorama to 2 CPUs and 4 GB memory.

[mschuricht]

50K objects is quite large. Are you actually using all of those objects?

This seems like a case for consolidation if possible to subnets and ranges where applicable as well as removing any unused objects.

Here is a script to identify unused objects to aid in a diagnosis: Unused and Duplicate Address Object Script

I would probably recommend moving to 5.1 for 64 bit support if you cannot reduce the number of objects. 5.1 requires 4 CPU cores and 4GB RAM at minimum. For a larger config it is recommended to move to 16GB of RAM. The release notes outline the recommended requirements as well as upgrade procedure if you decide to go this route.

[hkp]

I see what you did there... Trying to trick me into reading the documentation. 

They will all be used.  I upgraded to 5.1, but didn't actually read the release notes.  Heading off to do that now.

Thanks.

[kkeeton]

a quick example script for the Office Pro IP ranges.. this would be expanded as needed.

set tag Office365-OP

set address Office365-OP01 tag Office365-OP ip-netmask 65.52.98.231

set address Office365-OP02 tag Office365-OP ip-netmask 157.55.44.71

set address Office365-OP03 tag Office365-OP ip-netmask 157.55.160.109

set address-group Office365-OP tag Office365 dynamic filter 'Office365-OP'