topic Re: Confused over EBL size limit in General Topics

Confused over EBL size limit

networkadmin — Wed, 17 Aug 2016 18:07:00 GMT

We have a 3020 running 7.0.8 and are experimenting with MineMeld.

As soon as we get close to 5k IPs on the combined EBLs we get an error on a EBL refresh that it's been truncated as it's over the limit.

Palo Alto's own KB suggests that on an entry level PA-200 there is a limit of 50k items on all EBLs combined.

https://live.paloaltonetworks.com/t5/Learning-Articles/Working-with-External-Block-List-EBL-Formats-and-Limitations/ta-p/58795

Support are telling me that the limit on 3020 is 5k which doesn't seem to make sense as a) why would a 200 support more than a 3020 and b) what's the point of something like minmeld if you can only have 5000 IPs?

Any clarification would be great.

Re: Confused over EBL size limit

Greg_R — Wed, 17 Aug 2016 18:17:51 GMT

For 7.0.x and earlier read this article. Specifically the 3rd entry down.

https://live.paloaltonetworks.com/t5/General-Topics/Dynamic-Block-List-Limit-on-number-of-entries/m-p/100757/thread-id/44278

For 7.1 see this link.

https://live.paloaltonetworks.com/t5/General-Topics/Dynamic-Block-List-Limit-on-number-of-entries/m-p/100757/thread-id/44278

Either way PanOS sets aside 300 entries so the number will always be 300 lower than the maximum. The limit on your 3020 running 7.0.8 will be 4700.

Re: Confused over EBL size limit

lmori — Wed, 17 Aug 2016 21:32:28 GMT

In addition to Greg answer about enanchements in 7.1, note that:

- you can limit the number of entries of MineMeld output feeds retrieved by PAN-OS using the 'n' URL parameter. Example: https://<minemeld>/feeds/inboundhcfeed?n=1000 will retrieve only the top 1000 entries of the feed

- output feeds by default in MineMeld are sorted by recency. This means that when you retrieve the 1000 topmost entries, the 1000 most recent entries are retrieved

Regards,

luigi

Re: Confused over EBL size limit

Sistemas_SanLucar — Tue, 29 Nov 2016 10:55:57 GMT

Hi,

I have a Pa-500 version 6.1.0

show system state | match cfg.general.max-address

cfg.general.max-address: 0x9c4 --> 2500 IP
cfg.general.max-address-group: 0xfa --> 250
cfg.general.max-address-per-group: 0x1f4 --> 500
peer.cfg.general.max-address: 0x9c4
peer.cfg.general.max-address-group: 0xfa
peer.cfg.general.max-address-per-group: 0x1f4

I am confused with the limits of the lists. I can not predict how big the feed can become. If it's bigger than the limits, does the Palo Alto read the list as much as I can or can not read it?

With these values, how many lists can I have?
How many values can this list have?
In global, how many ip can I have?

Regards

Re: Confused over EBL size limit

lmori — Tue, 29 Nov 2016 11:50:17 GMT

Hi @Sistemas_SanLucar,

check here:

https://www.paloaltonetworks.com/documentation/61/pan-os/pan-os/policy/use-a-dynamic-block-list-in-policy.html

Your PA-500 with PAN-OS 6.1 can have:

- up to 10 Dynamic Block Lists

- each DBL can contain up to (max-addresses-300) entries = 2200 IPs

Note: this has changed in PAN-OS 7.1, check @Greg_R previous post.

In MineMeld you can use the "n" and "s" feed parameter to slice a feed. Example:

https://<minemeld ip>/feeds/inboundhc => full list of indicators

https://<minemeld ip>/feeds/inboundhc?n=2200 => first 2200 entries in the list

https://<minemeld ip>/feeds/inboundhc?s=2200&n=2200 => entries 2201-4400 in the list

Note: feeds by default are sorted based on the update time, this means that when you retrieve the first N entries these will be the N most recent entries.

Hope it helps,

luigi

Re: Confused over EBL size limit

Sistemas_SanLucar — Tue, 29 Nov 2016 15:53:01 GMT

I continue with the doubt. If the list has 5000 ip. What does Palo Alto do?

Does it only read from the list the ip that it allows?
Does it give error and does not read anything?

Thank you

Re: Confused over EBL size limit

Greg_R — Tue, 29 Nov 2016 15:57:15 GMT

If the list is larger than the firewall can support, it will download its max allowed (starting at the top and working down) and then drop anything longer than it can accomdate. At this point it will also throw a warning that the max limit has been hit. I'm trying to dig up the exact message, but I believe it was posted in the forums before.

Re: Confused over EBL size limit

Sistemas_SanLucar — Tue, 29 Nov 2016 16:14:17 GMT

Thank you.

Re: Confused over EBL size limit

kevin_thys — Wed, 08 Feb 2017 20:15:07 GMT

Hi Luigi,

I tried splitting the list but still getting error that maximum in the list is exceeded.

Config firewall:

xxx-Ransomware-IPv4-01 {
recurring {
hourly {
at 45;
}
}
url https://ip/feeds/xxx-Ransomware-IPv4?n=4600;
type ip;
description "Ransomware Minemeld list Medium confidence level";
}
xxx-Ransomware-IPv4-02 {
recurring {
hourly {
at 46;
}
}
url https://ip/feeds/xxx-Ransomware-IPv4?s=4600&n=4600;
type ip;
}
xxx-Ransomware-IPv4-03 {
recurring {
hourly {
at 47;
}
}
url https://ip/feeds/xxx-Ransomware-IPv4?s=9200&n=4600;
type ip;
}

Running lateste Minemeld and PAN-OS 7.0.9.

Error received when commit is done:

EBL(vsys1/xxx-Ransomware-IPv4-02) Exceeding max number of ips at line 4701

When I check in CLI it is starting at 4600 but not ending untill the end of the list.

I think that the parameter is n=4600 is not working.

Re: Confused over EBL size limit

lmori — Wed, 08 Feb 2017 21:25:40 GMT

Just tested this and works for me. Could you try this and paste the output ?

$ curl -s https://<minemeld>/feeds/inboundFeedMC\?s=4600\&n=4600  | wc
    4600    4600  133432

Thanks !