topic Re: Rest API and import in General Topics

Rest API and import

hkp — Tue, 27 Aug 2013 15:54:09 GMT

I need to add a lot of addresses (around 10,000) to my firewalls. I have them in a list and have created a script to add them one at a time. The problem is that this takes a long time. I then tried to add them all, but my URI was too long. So now I can add about 50 at a time. It still takes a while.

I also know that with the import command I can import a full config.

My question is this: Is there a way with the import command that I can add several addresses at once?

Thanks.

Re: Rest API and import

jvalentine — Tue, 27 Aug 2013 16:40:07 GMT

Is there a reason that you need to do this with the API? If this is a one-time event, you could do it pretty quickly through the CLI using the "scripting-mode" function. Here's what it looks like on a PA-4020 running 5.0.6 (via SSH console):

admin@pa4020> set cli scripting-mode on

admin@pa4020> configure

(now paste in the 10000 lines of objects)

admin@pa4020# set address object00001 ip-netmask 1.1.1.1

admin@pa4020# set address object00002 ip-netmask 1.1.1.2

...

admin@pa4020#

I also did the same thing on a PA-5050. The first 3000 will go pretty quick, and after that it will slow down quite a bit. The PA4020 was processing about 3-4 new objects/sec, while the PA5050 was adding about 7-8 entries/sec. Should be do-able in ~20 minutes if I had to extrapolate.

Re: Rest API and import

${userLoginName} — Wed, 28 Aug 2013 06:12:22 GMT

You could also export your configuration, add the objects manually in xml format and import again.

Shouldn't be hard to create a script to get the objects in PAN xml format:

for example:

<ip-netmask>10.1.1.1/32</ip-netmask>

</entry>

Or you can load a partial configuration, but I have no experience with this yet.

How to Load Partial Configs PAN-OS 4.0

Re: Rest API and import

hkp — Wed, 28 Aug 2013 20:51:51 GMT

This is what I finally did.

Now that I have all my addresses in Panorama adding a single address at a time takes over a minute.

Is that expected behavior?

Re: Rest API and import

mschuricht — Wed, 28 Aug 2013 21:17:23 GMT

Please answer the following questions:

How many objects do you have in total?
How many device groups do you have?
What is the RAM and CPU allocated to Panorama?

Re: Rest API and import

${userLoginName} — Thu, 29 Aug 2013 11:33:47 GMT

As mschuricht is implying, having a lot of objects in your configuration can cause an additional load / configuration latency on your device. When adding an address, the device will parse the entire configuration file to check if the objects already exists etc.

Re: Rest API and import

hkp — Thu, 29 Aug 2013 16:17:53 GMT

Fair enough.

I have over 50,000 objects and two device groups.

I just increased panorama to 2 CPUs and 4 GB memory.

Re: Rest API and import

mschuricht — Thu, 29 Aug 2013 17:43:21 GMT

50K objects is quite large. Are you actually using all of those objects?

This seems like a case for consolidation if possible to subnets and ranges where applicable as well as removing any unused objects.

Here is a script to identify unused objects to aid in a diagnosis: Unused and Duplicate Address Object Script

I would probably recommend moving to 5.1 for 64 bit support if you cannot reduce the number of objects. 5.1 requires 4 CPU cores and 4GB RAM at minimum. For a larger config it is recommended to move to 16GB of RAM. The release notes outline the recommended requirements as well as upgrade procedure if you decide to go this route.

Re: Rest API and import

hkp — Thu, 29 Aug 2013 18:26:51 GMT

I see what you did there... Trying to trick me into reading the documentation.

They will all be used. I upgraded to 5.1, but didn't actually read the release notes. Heading off to do that now.

Thanks.

Re: Rest API and import

kkeeton — Sat, 20 Dec 2014 00:10:51 GMT

a quick example script for the Office Pro IP ranges.. this would be expanded as needed.

set tag Office365-OP

set address Office365-OP01 tag Office365-OP ip-netmask 65.52.98.231

set address Office365-OP02 tag Office365-OP ip-netmask 157.55.44.71

set address Office365-OP03 tag Office365-OP ip-netmask 157.55.160.109

set address-group Office365-OP tag Office365 dynamic filter 'Office365-OP'