Dynamic Address Group (DAG) PAN-OS / DAGPusher prototype

[danilo.souza]

Hi guys,

I'm trying to use the the DAGPusher prototype but, unhappily, I'm dealing with some problems. May be some of you could help me with it.

My scenario:

I use a generic miner to extract IPv4 (/32) from a specific location (that is working). Then it is sent to the DAGPusher node (that is working). Now I want to push them to Firewall/Panorama (it is not working).

My configuration:

In my local DAGPusher prototype, I use the configuration showed in figure 1 (green arrow). You can see that I only set the "tag_prefix":"agencias" parameter.

The status tab (figure 3) in the node cloned from this prototype, I have the indicators been received. Fine! And the Handled_Device tab show the Firewall destination (figure 2). Although I have multiplus devices and manage all of them through Panorama I choose to test DAGPusher with only one of them, at first.

Monitoring the traffic log I can see that my MM VM is trying to communicate with the device, and this traffic is allowed.

In Panorama I created a shared DAG with only one match: 'agencias' (figure 3).

Status/Questions:

At this point my DAG is not populated. Through the CLI, using the command "show object registered-ip all" I got nothing.
Could you guys identify something wrong? The syntax and the tags I used are corrects?
The fact that I created a shared DAG in Panorama could be the problem? Can I populate a shared DAG only in one Firewall?

Thank you in advanced.
Best regards.

[danilo.souza]

Hi guys,

I would like to share some information and ask one more question. In the figure attached, you can see that my MM is pushing data to the Firewall (traffic log). My question is: EDLs have a specific interface to be received (DEVICE -> SERVICES -> SERVICES ROUTE CONFIGURATION). How about DAGs? Do the API in DAGpusher have to send the data to a specific interface?

Best regards.

[danilo.souza]

Hi @lmori , @xhoms

Could you help me with this issue. I believe the problem is in my local miner, specifically the way the "tag" is attached to the indicator.

My shared DAG in Panorama is prepared to match 'mm_loc_agencias'.

My local DAGPusher has, now, this configuration:

==============

{

    "tag_prefix": "mm_",

    "tag_watermark": "pushed",

    "tag_attributes": "loc"

}

==============

My local miner is from the class "minemeld.ft.http.HttpFT". I've tried out three configurations so far and none of them successfully. The basic configuration is:

==============

age_out:
    default: null
    sudden_death: true
attributes:
    confidence: 95
    share_level: green
    type: IPv4
    interval: 300

    tag:
indicator:
    regex: ([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})
    transform: \1
verify_cert: false
source_name: xxx_log
url: http://xxx/static/log.txt

================

In tag I've tried:

================

    tag:

        "loc": "agencias"

------

    tag:

        loc: "agencias"

-------

    tag: agencias

================

I would really appreciate any help. I've read the "Poor man's NAC Application" article, but there, you use a localDB node to feed the DAGPusher. In my case I use a local miner, and the way the attributes are attached to indicator are different.

Thank you in advanced.

Best regards.

[xhoms]

Hi @danilo.souza,

access to PANOS API is not controlled with service route configurations but:

• The user/password you're using might have a RBAC policy denying access to the API (user-id)
• You could be using VSYS (although it is supported you must provide it into the prototype)

[danilo.souza]

Hi @xhoms

thank you for your reply. Almost sure the credential used has the privilege to access the API. I will double check it as soon as possible. To be frank, I am most concerned with the sintaxe and tags used in the Miner and DAGPusher.

Can you confirm that the way I explicity the "tag" (at least one of the three ways I've tried out) atribute in my Miner is correct? Will it, togheter with the "tag_prefix" (mm_) in DAGPusher, match the 'mm_loc_agencias' criteria that I put in the shared DAG? I'm not sure.

Can you confirm that is possible to populate a shared DAG (created in Panorama) just in one Firewall?

Thank you again

Best regards

[xhoms]

Hi @danilo.souza,

regardless of your miner and output node configurations, any IPv4/32 indicator reaching (Update message) the DAGOutput node should register the IP using the pre-defined watermark tag. But you report that the command "show object registered-ip all" does not return anything.

This is what is making me believe that either the user does not have access to the API or you're using VSYS and a csutom configuration is required.

[danilo.souza]

Hi @xhoms

I double checked the credentials and I can confirm that the user used to push the IPs to the DAG has the correct permissions. Could you tell me wich specific procedures should I follow in case I have a VSYS ( only one, named vsys1, with default configurations)?

Thank you again.

Best regards.

[danilo.souza]

Hi @xhoms

This customization that you said (dealing with a VSYS), is something similar the hacking we apply when using an api key (DAGPusher file configuration)? The same way you mentioned in the "Using MineMeld to implement a poor man's NAC application (DAGPusher)" article? I couldn't find any information.

[xhoms]

Hi @danilo.souza,

probably is time to troubleshoot the PANOS device. Palo Alto Networks TAC team can support you.

What I'd do is to turn on debug messages for user-is ip registration events:

admin@VM-Series> debug user-id set userid regip
admin@VM-Series> debug user-id on debug

And then tail the useridd.log file

admin@VM-Series> tail follow yes mp-log useridd.log 

A typical successfull registration of an IP would generate a log track record like the following one

2018-09-19 22:53:01.330 -0700 debug: cfgagent_opcmd_callback(pan_cfgagent.c:468): useridd: cfg agent received op command from server
2018-09-19 22:53:01.331 -0700 debug: cfgagent_doop_callback(pan_cfgagent.c:503): received signal to execute for agent: useridd
2018-09-19 22:53:01.336 -0700 debug: pan_regip_add_ip(pan_reg_ip.c:1080): add registered ip 172.16.214.200 in vsys 1
2018-09-19 22:53:01.340 -0700 debug: pan_regip_reg(pan_reg_ip.c:1186): reg ip 172.16.214.200 with tag WebAppServer in vsys 1
2018-09-19 22:53:01.978 -0700 Processing dnld delta : 1, full : 2
2018-09-19 22:53:01.982 -0700 debug: pan_regip_notify_modified(pan_reg_ip.c:3153): regip-modified notified to other daemons as: incremental change
2018-09-19 22:53:01.982 -0700 dnld 1 registered ip takes 0 seconds

And, a typical unregistration like this

2018-09-19 22:53:44.620 -0700 debug: cfgagent_opcmd_callback(pan_cfgagent.c:468): useridd: cfg agent received op command from server
2018-09-19 22:53:44.621 -0700 debug: cfgagent_doop_callback(pan_cfgagent.c:503): received signal to execute for agent: useridd
2018-09-19 22:53:44.621 -0700 debug: pan_regip_obj_remove_tag(pan_reg_ip.c:1027): unregister ip 172.16.214.200 in vsys 12018-09-19 22:53:44.621 -0700
debug: pan_regip_unreg(pan_reg_ip.c:1365): unreg ip 172.16.214.200 with tag WebAppServer in vsys 1
2018-09-19 22:53:44.621 -0700 debug: pan_regip_unreg(pan_reg_ip.c:1410): remove registered ip 172.16.214.200 in vsys 1 since no tags associated with i
t any more

[danilo.souza]

Hi @xhoms

I think I have to backtrack when I said I had the correct credentials. At first, I used a key generated for a active "super user" because I was just testing DAGPusher and didn't want to create a new user. Everything I reported in previous posts were related with this user. To eliminate any doubt I tried to push the IP to Firewall by myself using the line below:

--------------

curl -F key=LUFRPT0-----------kJQbm9qTT0 --form file=@/home/user/ips.xml "https://firewall/api/?type=user-id"

--------------

But I got this answer:

--------------

<response status = 'error' code = '403'><result><msg>Invalid credentials.</msg></result></response>

--------------

So I created a new user (now, not a "super user", but a "panorama administrator" (PANORAMA) and "device administrator" (FIREWALL)) and submit the previous line with the new key:

--------------

curl -F key=LUFRPT1-----------sc21JVT0 --form file=@/home/user/ips.xml "https://firewall/api/?type=user-id"

--------------

and I got the same answer:

--------------

<response status = 'error' code = '403'><result><msg>Invalid credentials.</msg></result></response>

--------------

Both usear are capable of create DAG, adresses (Objects) etc. and commit these changes. Does it make any sense? 

[xhoms]

Hi @danilo.souza,

is it possible that you're targetting a Panorama instead of a PANOS device to push User-ID REGISTER messages?

Panorama does not implemente the User-ID API. That is a PANOS only feature. You can use Panorama as an API Gateway to reach the PANOS Device API. But, in any case, the REGISTER/UNREGISTER messages generated by the DAG Pusher output node must be targetted to a PANOS Device (or to a list of devices)

[danilo.souza]

Hi @xhoms

No, I'm not targeting Panorama. I created the shared DAG in Panorama, but I'm pushing the IP to a specific Firewall. I used the command below (from the MM machine) to generate the key (for both users).

-----------------

curl -k -X GET 'https://firewall/api/?type=keygen&user=user&password=xxxxxxx'

----------------

After your comment, I check out if Panorama would generate a different api key (for both users), so I repeated the proccess.

-----------------

curl -k -X GET 'https://panorama/api/?type=keygen&user=user&password=xxxxxxx'

----------------

I got the same keys. So the keys are double checked.

[xhoms]

Hi @danilo.souza,

I'm afraid you'll have to sort out the access to the UserID entry point in the PANOS Device API with Palo Alto Networks TAC before trying to deploy the DAG Pusher node in MineMeld

[danilo.souza]

Hi @xhoms

I opened a ticket at PaloAlto and resolved the problem with the api key ("Invalid Credentials").

Inicially, I generated my keys through a Curl command from the MM machine. With the PaloAlto support, I used the browser and can confirm that the keys are barely different. I don't want speculate but I think my problem was an issue with Curl when generating my keys.

Later, I can detail the procedure followed to generate the new keys.

But, the problem is not resolved yet. I can't populate my DAG using the DAGPusher, but I can do that manually. In the figure below you can see that I uploaded two IPs. To do that, I used (in the browser ) the line below:

----------

https://PANORAMA/api/?key=LUFRPT04OG---------VkJQbm9qTT0=&cmd=<uid-message><version>2.0</version><type>update</type><payload><register><entry ip="152.XXX.XXX.121"><tag><member>mm_loc_agencias</member></tag></entry></register></payload></uid-message>&type=user-id&target=FIREWALL_SERIAL_NUMBER

-----------

DAG populated

Using the same user (login/password, not api key) in MM, I can't see new IPs populating my DAG. My indicator in the DAGPusher node is showed in the figure below.

Indicator in DAGPusher

I am using the tag prefix "mm_" in DAGPusher and the miner attachs the tag "loc_agencias" to the indicator. My DAG is prepared to match "mm_loc_agencias".

Can you visualize any error in the precedures/parameters I am following/setting in my configuration.

Best regards.

[xhoms]

Hi @danilo.souza,

could you go back to message https://live.paloaltonetworks.com/t5/MineMeld-Discussions/Dynamic-Address-Group-DAG-PAN-OS-DAGPusher... and execute the debug commants shown there to troubleshoot the UserID API from PANOS' foint of view?