Check logs for large number of IP Addresses

[SSargent_ICTWA]

I received a list of over 600 IP addresses associated with a botnet from a reliable threat intelligence source. I would like to check our logs for traffic to or from these addresses but creating a filter with that many IP addresses seems unwieldy. Does anyone know a better way?

[murali438]

You can export the logs to a CSV format and use local search.

By default it exports 2,000 rows but you can change it.

Export Logs (paloaltonetworks.com)

[TomYoung]

Hi @SSargent_ICTWA ,

The PANW free product MineMeld was build to easily incorporate threat intelligence feeds into the firewall.  https://www.paloaltonetworks.com/products/secure-the-network/subscriptions/minemeld

If your 600 IP addresses can be pulled from a simple HTML list off the Internet, you could create an EDL directly to it without having to use MineMeld.  https://docs.paloaltonetworks.com/pan-os/10-1/pan-os-admin/policy/use-an-external-dynamic-list-in-po...

Once you have the EDL configured (directly or from MineMeld) you can do a lot more:

1. Automatically block the traffic.
2. Generate reports on IP addresses that hit the block rule.
3. Eliminate a lot of manual work in threat hunting your botnets!

Thanks,

Tom

[SSargent_ICTWA]

@murali438 Thanks for that suggestion. I hadn't thought of that, and I gave it a try. Unfortunately, 1 million records are only enough to get a snapshot of a couple of hours on our network. I need to search all the logs that the firewall has at once. This may help me in other ways, though, so I appreciate the idea.

[SSargent_ICTWA]

@TomYoung This is a good suggestion. I do have some EDL's configured. In this case, the threat intelligence came in a CSV file attached to an email. I put MineMeld on my roadmap, though, for future testing and possible implementation. It's just a little more involved than justified for this occasional intelligence source.

[TomYoung]

Hi @SSargent_ICTWA ,

You are correct.  Minemeld is involved.  If you have an internal web server, it may be easier to convert the CSV to a simple HTML page and point an EDL to it.

Thanks,

Tom

[SSargent_ICTWA · Accepted solution]

After trying a few different methods, I used Excel formulas to create a traffic log filter with all 600 addresses. I used the filter directly in the UI and it ran in less than three minutes. Not as unwieldy as I had imagined.

[SSargent_ICTWA]

As a follow up to the suggestion about Minemeld... that's a minefield. Palo Alto "open sourced" it for the developer community to support over a year ago, and it has not received developer community support as far as I can tell. There are no longer any prebuilt Minemeld VM's as promised on the page that is linked in a previous comment. Because it is not supported, the Minemeld codebase is now dependent on outdated, vulnerable Linux components. What a disappointment.

In the meantime, Palo Alto is eager to sell us a single Cortex license for over $100,000 to supposedly fill this need (without the artificial limitations of the community edition).

I just need automated TI EDL's. Neither of these solutions are satisfactory.