topic Re: Check logs for large number of IP Addresses in Next-Generation Firewall Discussions

Check logs for large number of IP Addresses

SSargent_ICTWA — Wed, 09 Nov 2022 16:06:20 GMT

I received a list of over 600 IP addresses associated with a botnet from a reliable threat intelligence source. I would like to check our logs for traffic to or from these addresses but creating a filter with that many IP addresses seems unwieldy. Does anyone know a better way?

Re: Check logs for large number of IP Addresses

murali438 — Fri, 11 Nov 2022 10:20:37 GMT

You can export the logs to a CSV format and use local search.

By default it exports 2,000 rows but you can change it.

Export Logs (paloaltonetworks.com)

Re: Check logs for large number of IP Addresses

TomYoung — Fri, 11 Nov 2022 13:52:57 GMT

Hi @SSargent_ICTWA ,

The PANW free product MineMeld was build to easily incorporate threat intelligence feeds into the firewall. https://www.paloaltonetworks.com/products/secure-the-network/subscriptions/minemeld

If your 600 IP addresses can be pulled from a simple HTML list off the Internet, you could create an EDL directly to it without having to use MineMeld. https://docs.paloaltonetworks.com/pan-os/10-1/pan-os-admin/policy/use-an-external-dynamic-list-in-policy/external-dynamic-list

Once you have the EDL configured (directly or from MineMeld) you can do a lot more:

Automatically block the traffic.
Generate reports on IP addresses that hit the block rule.
Eliminate a lot of manual work in threat hunting your botnets!

Thanks,

Tom

Re: Check logs for large number of IP Addresses

SSargent_ICTWA — Mon, 14 Nov 2022 20:38:25 GMT

@murali438 Thanks for that suggestion. I hadn't thought of that, and I gave it a try. Unfortunately, 1 million records are only enough to get a snapshot of a couple of hours on our network. I need to search all the logs that the firewall has at once. This may help me in other ways, though, so I appreciate the idea.

Re: Check logs for large number of IP Addresses

SSargent_ICTWA — Mon, 14 Nov 2022 20:44:01 GMT

@TomYoung This is a good suggestion. I do have some EDL's configured. In this case, the threat intelligence came in a CSV file attached to an email. I put MineMeld on my roadmap, though, for future testing and possible implementation. It's just a little more involved than justified for this occasional intelligence source.

Re: Check logs for large number of IP Addresses

TomYoung — Mon, 14 Nov 2022 21:03:48 GMT

Hi @SSargent_ICTWA ,

You are correct. Minemeld is involved. If you have an internal web server, it may be easier to convert the CSV to a simple HTML page and point an EDL to it.

Thanks,

Tom

Re: Check logs for large number of IP Addresses

SSargent_ICTWA — Mon, 14 Nov 2022 21:05:42 GMT

After trying a few different methods, I used Excel formulas to create a traffic log filter with all 600 addresses. I used the filter directly in the UI and it ran in less than three minutes. Not as unwieldy as I had imagined.

Re: Check logs for large number of IP Addresses

SSargent_ICTWA — Wed, 08 Mar 2023 15:20:44 GMT

As a follow up to the suggestion about Minemeld... that's a minefield. Palo Alto "open sourced" it for the developer community to support over a year ago, and it has not received developer community support as far as I can tell. There are no longer any prebuilt Minemeld VM's as promised on the page that is linked in a previous comment. Because it is not supported, the Minemeld codebase is now dependent on outdated, vulnerable Linux components. What a disappointment.

In the meantime, Palo Alto is eager to sell us a single Cortex license for over $100,000 to supposedly fill this need (without the artificial limitations of the community edition).

I just need automated TI EDL's. Neither of these solutions are satisfactory.