pandevice newbie issues with fetching objects (addressobject, addressgroup, serviceobject...

L1 Bithead

pandevice newbie issues with fetching objects (addressobject, addressgroup, serviceobject...

Hello,

 

I'm not pretending to be a pandevice guru... moreover, I'm not pretending to be python guru. Rather basic skills in both. I've got a task to retrieve firewall configuration (rules + object definitions) from two datacenters (primary & DR, separate setups), store this in some reasonable format and compare to find differences and make sure that they are in sync (or at least to make sure that differences are related to different IP scheme and some minor functionality differences - in DR we have some additional test environment).

 

High level setup is that in both DC we have Panoramas and PA firewalls managed by those Panoramas. We use Device-Groups in Panoramas and several vsys in PA FW. We use both physical PA FW and virtual appliances (private cloud). Device-Groups include at least 3 levels of inheritance (Shared > DataCenter > areaX, Shared > privcloud > areaY and similar)

 

We decided that using pandevice might help us with fetching required information and store rule&object definitions in JSON files then use some handcrafted tool to make 'smart diff' (checking some basic rule details, like focusing on source/dest/source zone/dest zone/service). For some reasons Python 2.7 is being used. However, using 3.7 does not make any difference (tried).

 

I'm able to fetch prerulebase and postrulebase, connecting to all interesting Device-Groups. I can build a complete ruleset for each firewall based on those rulebases and checking target devices included in rules. But I'm not able to fetch any addressobject/addressgroup or service objects. We are creating objects mostly in DataCenter device group (see scheme above) and I see them in Panorama or by looking at config via CLI, but when I try to fetch them by using pandevice I get 0.

 

I'm using this to fetch rulebases (working, Py27):

from __future__ import print_function, division
import sys
import io
import json
import pandevice
# from pandevice import base
# from pandevice import firewall
from pandevice import panorama
from pandevice import objects
from pandevice import policies

DEVICES = ["a.b.c.d", "e.f.g.h"]   # one Panorama in main DC and the other in DR DC
DEVICEGROUPS = ["Shared", "DataCenter", "areaX", "areaY"]   # Shared > DataCenter > areaX, Shared > DataCenter > areaY
PAN_API_KEY = "xxxxxxxapikeyxxxxx"

for device in DEVICES:
    pano = panorama.Panorama(device, api_key=PAN_API_KEY)
    for devgrp in DEVICEGROUPS:
        panogrp = panorama.DeviceGroup(devgrp)
        pano.add(panogrp)

        # PreRulebase
        # Build the object tree
        prb = policies.PreRulebase()
        panogrp.add(prb)
        # Fetch rules
        all_rules = policies.SecurityRule.refreshall(prb)
        if all_rules:
            combined_list = []
            iteration = 1
            for element in all_rules:
                # iteration - starting with 1, not 0
                # Populate dictionary mappings
                i = {
                    '_seq_': iteration,
                    '_name_': element.name,
                    'fromzone': element.fromzone,
                    'tozone': element.tozone,
                    'source': element.source,
                    'source_user': element.source_user,
                    'destination': element.destination,
                    'application': element.application,
                    'service': element.service,
                    'category': element.category,
                    # ... etc, enumerated fields that are interesting for our analysis
                }
                iteration += 1
                combined_list.append(i)
            # dumping ALL rules to JSON file, make sure that we use correct UTF-8 encoding and proper
            # line-endings (LF only, unix-like)
            with io.open("json/" + device + "_" + devgrp + "_prerulebase_processed.json",
                         "w", newline="\n", encoding="utf8") as outfile:
                data = json.dumps(combined_list, sort_keys=True, indent=4, separators=(',', ': '),
                                  ensure_ascii=False)
                outfile.write(unicode(data))
                outfile.write(unicode('\n'))

        # PostRulebase
        # Build the object tree
        prb = policies.PostRulebase()
        panogrp.add(prb)
        # Fetch rules
        all_rules = policies.SecurityRule.refreshall(prb)
        if all_rules:
                  #
                  # the same approach as in PreRulebase - get fields, output to JSON
                  #

After running above code I'm getting several JSON files with prerulebases and postrulebases for each defined Device Group, for each of two listed Panoramas. Actually this code comes from various pandevice examples and from some discussions here in this forum.

 

Now, similar approach for address/service objects:

 

...
    #
    # Get Objects for each listed device group
    for devgrp in DEVICEGROUPS:
        panogrp = panorama.DeviceGroup(devgrp)
        pano.add(panogrp)
        #
        # AddressObjects
        aobjects = pandevice.objects.AddressObject()
        pano.add(aobjects)
        current_object_list = pandevice.objects.AddressObject.refreshall(pano)
        # (tried also pandevice.objects.AddressGroup.refreshall(aobjects)
        objects_list = []
        for object_element in current_object_list:
            obj = {
                'name': object_element.name,
                'value': object_element.value,
                'type': object_element.type,
                'description': object_element.description,
                'tag': object_element.tag
            }
            objects_list.append(obj)
       if objects_list:
with io.open("json/" + device + "_" + devgrp + "_addressobjects.json",
"w", newline="\n", encoding="utf8") as outfile:
data = json.dumps(objects_list, sort_keys=True, indent=4, separators=(',', ': '),
ensure_ascii=False)
outfile.write(unicode(data))
outfile.write(unicode('\n'))
# # AddressGroups agobjects = pandevice.objects.AddressGroup() pano.add(agobjects) current_object_list = pandevice.objects.AddressGroup.refreshall(pano) # (tried also pandevice.objects.AddressGroup.refreshall(agobjects) objects_list = [] for object_element in current_object_list: obj = { 'static_value': object_element.static_value, 'dynamic_value': object_element.dynamic_value, 'description': object_element.description, 'tag': object_element.tag } objects_list.append(obj)
if objects_list:
with io.open("json/" + device + "_" + devgrp + "_addressgroups.json",
"w", newline="\n", encoding="utf8") as outfile:
data = json.dumps(objects_list, sort_keys=True, indent=4, separators=(',', ': '),
ensure_ascii=False)
outfile.write(unicode(data))
outfile.write(unicode('\n'))

I've tried several combinations when refreshing objects but with no luck. Everytime I'm just getting nothing, empty lists.

 

I probably do something obviously wrong but I have no experience to find out what.

 

Any hints? Any help here?

 

Thanks!

Arek

L4 Transporter

Re: pandevice newbie issues with fetching objects (addressobject, addressgroup, serviceobject...

The problem is your pandevice object hierarchy tree.  When you are getting security rules, your tree needs to look like this:

 

Panorama > device group > rulebase > (security rules)

 

For address objects, it should be this:

 

Panorama > device group > (address objects)

 

So in your second script, you're invoking pandevice.objects.AddressGroup.refreshall(aobjects), but you should using the device group devgrp as the param passed in to the address objects/group's refreshall().

 

If you look at the CHILDTYPES for each object, you'll see which objects can be children of that object, so that is the way to see what goes with what.

 

One other final tweak, you do not need to manually turn pandevice objects into JSON, there is an .about() every object has that you should use which will give you a dict of the params.

 

L1 Bithead

Re: pandevice newbie issues with fetching objects (addressobject, addressgroup, serviceobject...

Thank you so much!

 

Changing 'scope' helped.

 

for device in DEVICES:
    pano = panorama.Panorama(device, api_key=PAN_API_KEY)

    #
    # when you need to fetch SHARED objects (common to all Device Groups), like AddressObject, AddressGroup, ServiceObject, ServiceGroup
    current_object_list = pandevice.objects.AddressObject.refreshall(pano)
    objects_list = []
    for object_element in current_object_list:
        obj = {
            'name': object_element.name,
            'value': object_element.value,
            'type': object_element.type,
            'description': object_element.description,
            'tag': object_element.tag
        }
        objects_list.append(obj)

    #
    # when you need to fetch objects in the Device Group
    for devgrp in DEVICEGROUPS:
        panogrp = panorama.DeviceGroup(devgrp)
        pano.add(panogrp)
        current_object_list = pandevice.objects.AddressObject.refreshall(panogrp)
        objects_list = []
        for object_element in current_object_list:
            obj = {
                'name': object_element.name,
                'value': object_element.value,
                'type': object_element.type,
                'description': object_element.description,
                'tag': object_element.tag
            }
            objects_list.append(obj)

Now I'm able to retrieve interesting objects and build a complete rulesets.

 

If I'm not mistaken, the same rule is for rulesets - common prerulebase/postrulebase (ie, seen as Shared). We don't have any 'Shared' rules so I can't confirm that.

 

Thanks! :)

L1 Bithead

Re: pandevice newbie issues with fetching objects (addressobject, addressgroup, serviceobject...

btw

I'm not quite sure what do you mean by using "about()". Is just about getting schema for objects? or, can I use it to dump list of objects?

Something like this?

        current_object_list = pandevice.objects.AddressGroup.refreshall(panogrp)
        objects_list = []
        for object_element in current_object_list:
            objects_list.append(object_element.about())
Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the Live Community as a whole!

The Live Community thanks you for your participation!