A few questions

[jtrevaskis]

Afternoon

Firstly I want to say I really like this product, it has endless possibilities in improving internal security in our environment.

I have a few questions I hope you can help me clarify so I understand how to use the product better.

I am using a syslog miner to send syslog TRAFFIC and THREAT data to Mine Meld from my Paloalto firewall. 

1.  When I look on the miner logs I see threats being recorded, then shortly after an hour later I see a 'EMIT_WITHDRAW' log entry. If I look in the feed connected to the miner, I see the IP address in the EMIT_WITHDRAW is removed the the feed.

Are you able to explain how this aging process works and how I can keep IP's in the feed longer? I'd like 7 or 30 days instead of an hour.

2. Is it possible to have multiple syslog miners for incoming PA events? I'd like to use multiple miners so I can process threat and traffic events seperately. Also I'd like to control confidence of events individually which seems to require seperate miners that can feed into seperate feeds. 

Perhaps I am approaching this wrong, any advice would be good.

3. If I trigger a threat type event on my PA, I recieve the threat event in my miner. Also If I redirect the rsyslog to a file, I see that threat info in a file. 

However for traffic data, I see it in a syslog redirect file, but I never see traffic data in the miner. I also see no data in the stats/syslog section of the miner

Is there anything specific I need to do to use traffic data? 

I have a basic rule setup for traffic.

conditions:
- type == 'TRAFFIC'
fields:
- src_ip
indicators:
- src_ip

Thanks for any assistance or advice you can provide

[xhoms]

Does your PANOS device run release 8.0? MineMeld 0.9.42 introduced a new miner class ( minemeld.ft.localdb.Miner ) exposed through the stdlib.localDB prototype that can be used to accept indicators from any system that can forward alerts using a RESTful API. And PANOS 8.0 introduced the feature called HTTP Log Forwarding that fits into it.

This combination provides you with a unique way to bind PANOS devices with MineMeld as explained in the section 5 or the article Using MineMeld as a Incident Response Platform. 

You can use PANOS log forwarding profiles that create JSON documents out of log fileds and send them to MineMeld. This way the source specifies attributes like "ttl" (aging time in seconds), "confidence" and "share_level". A single localDB miner could be used by many log forwarding profiles and honor the aging provided by each of them

[jtrevaskis]

Thanks this seems like a great solution and I do have Palo 8.0.x running

I am getting a 401 unauthorized though when using minemeld admin credentials, is this something I am doing wrong?

[2017-09-08 02:24:04 UTC] [1327] [INFO] AUDIT - {"msg": null, "action": "POST /config/data/testminer_indicators/append", "params": [["value:t", ["localdb"]], ["jsonbody", "{\"tty\": 7200, \"share_level\": \"green\"}"]], "user": "mm-anonymous"}

127.0.0.1 - - [08/Sep/2017:02:24:04 +0000] "POST /config/data/testminer_indicators/append?t=localdb HTTP/1.0" 401 12 "-" "-"

[xhoms]

@jtrevaskis : Yes. I've also experienced this. Looks like an issue in PANOS 8.0. I've already filled a technical support case. In the meanwhile you can use the following workaround:

• Use https://www.blitter.se//utils/basic-authentication-header-generator/ to generate a value for your basic authentication. In the case of admin/minemeld the value should be Basic YWRtaW46bWluZW1lbGQ= (including the "Basic ").
• Add a new header called "Authorization" with that value in the Payload section

[jtrevaskis]

thanks ill try that and get back to you

[jtrevaskis]

There must be something wrong with my payload, I've played/changed with this 30 times and still get a similar output.

The data flows, but just arrives at Mine Meld with additional \" etc

Any suggestion you can give would be much appreciated

[xhoms · Accepted solution]

Humm ...

which browser are you using to access the PANOS device? It looks like your browser is escaping the double quotes you write in the Payload form/textArea.

You can give the CLI a try (the content in my example)

admin@PA-VM> configure
Entering configuration mode
[edit]                                                                                                                                                                                
admin@PA-VM# edit shared log-settings http minemeld format wildfire 
[edit shared log-settings http minemeld format wildfire]                                                                                                                              
admin@PA-VM# set payload '{"type":"sha256","indicator":"$filedigest","share_level":"green","ttl":3600}'

[jtrevaskis]

I am using Chrome

I have not retried JSON format yet, but I will shortly

I got this working with plain text so far

[jtrevaskis]

Thanks again, this is working great as a threat sharing solution for me between PA and OpenDXL

[m_matthey]

@xhoms For you case the issue is still present with PAN OS 8.1.6 under Panorama

Works fine with the header "Authorization"