Azure ip-range list EDL size

[MarcusHil]

Hi,

I ran into a problem today when expanding a customer's environment. I'd previously set up an EDL pointing to a Minemeld-generated list of all Azure ip-ranges, no problem thus far. I've done this for other customers before without any issue but noticed now that when I used the recommended prototype azure.cloudIPsWithServiceTags it generated a list with some 24000 rows of ip ranges whereas the old one I've used only generated in the region of 3000. So as I expanded the security policies and NAT rules with more references to the EDL, I got this message when pushing the config from Panorama:

Details:

. Error: Failed to get vsys config, already allocated (131072 bytes)

. failed to handle CONFIG_UPDATE_START

. (Module: device)

. Commit failed

Which from as best I can gather is down to the config-size growing too large for the VM300's. Anyone here run into the same problem? Or how do you best get around this issue? Set filters to exclude all irrelevant ip-ranges? I should perhaps add that this would be a general rule for all Azure VMs regardless of region to be able to speak directly to Azures backbone services and differentiate it from general internet access so they can access things like Windows Update, activate windows licenses, update Linux VMs etc.

[JoergSchuetter]

Hello

The IP addresses and networks from the Azure servicetag have overlapping networks. Do you see a way to consolidate all IPs first?

I don't use minemeld, simply use python for my automation tasks.

In python you would loop over all networks, add them to your bucket

bucket = netaddr.IPSet()

for ip in ... :

    bucket.update(netaddr.IPSet([ip, ]))

extract the consolidated networks

for net in bucket.iter.cidrs():

    print(net.__str__())

[CSavoy · Accepted solution]

Hi Marcus,

I'm new with Minemeld and never used the old Azure Miner. 

But I found the list we get using the new miner very big and investigated further.

The solution I found is to set a filter in the Minemeld processor, selecting only the prefixes having a null value "" into the azure_system_service_list" :

This way you get all the "AzureCloud" prefixes, which should be like the old miner...

If that's not enough, you can also play with some "Regional" filters, like this :

NB : Do not use this basic filter 

  - azure_region == 'uksouth'

Because most prefixes appears twice in the .json file, one in the regional section, and a second time in the 'null' section at the end. And the default Miner retains only the last value it sees, i.e (view of the log) :

[MarcusHil]

Very cool, I'll give that a try but that looks very much like it would work. Thanks!

[MarcusHil]

Hi Christophe,

Where do you see the output in your third screenshot (with the arrows)? Try as I might I can't seem to find it in the logs...

[MarcusHil]

Nevermind, two seconds after posting I found it 😄