Azure ip-range list EDL size

Announcements

Changes to the LIVEcommunity experience are coming soon... Here's what you need to know.

Reply
MarcusHil
L1 Bithead

Azure ip-range list EDL size

Hi,

I ran into a problem today when expanding a customer's environment. I'd previously set up an EDL pointing to a Minemeld-generated list of all Azure ip-ranges, no problem thus far. I've done this for other customers before without any issue but noticed now that when I used the recommended prototype azure.cloudIPsWithServiceTags it generated a list with some 24000 rows of ip ranges whereas the old one I've used only generated in the region of 3000. So as I expanded the security policies and NAT rules with more references to the EDL, I got this message when pushing the config from Panorama:

Details:

. Error: Failed to get vsys config, already allocated (131072 bytes)

. failed to handle CONFIG_UPDATE_START

. (Module: device)

. Commit failed

 

Which from as best I can gather is down to the config-size growing too large for the VM300's. Anyone here run into the same problem? Or how do you best get around this issue? Set filters to exclude all irrelevant ip-ranges? I should perhaps add that this would be a general rule for all Azure VMs regardless of region to be able to speak directly to Azures backbone services and differentiate it from general internet access so they can access things like Windows Update, activate windows licenses, update Linux VMs etc.

 


Accepted Solutions
Christophe_Savoy
L1 Bithead

Hi Marcus,

 

I'm new with Minemeld and never used the old Azure Miner. 

But I found the list we get using the new miner very big and investigated further.

 

The solution I found is to set a filter in the Minemeld processor, selecting only the prefixes having a null value "" into the azure_system_service_list" :

 

Christophe_Savoy_0-1612455781625.png

 

This way you get all the "AzureCloud" prefixes, which should be like the old miner...

 

If that's not enough, you can also play with some "Regional" filters, like this :

 

Christophe_Savoy_1-1612456096905.png

 

 

NB : Do not use this basic filter 

  - azure_region == 'uksouth'

 

Because most prefixes appears twice in the .json file, one in the regional section, and a second time in the 'null' section at the end. And the default Miner retains only the last value it sees, i.e (view of the log) :

Christophe_Savoy_2-1612456351929.png

 

 

View solution in original post


All Replies
JoergSchuetter
L4 Transporter

Hello

The IP addresses and networks from the Azure servicetag have overlapping networks. Do you see a way to consolidate all IPs first?

I don't use minemeld, simply use python for my automation tasks.

In python you would loop over all networks, add them to your bucket

bucket = netaddr.IPSet()

for ip in ... :

    bucket.update(netaddr.IPSet([ip, ]))

 

extract the consolidated networks

for net in bucket.iter.cidrs():

    print(net.__str__())

Christophe_Savoy
L1 Bithead

Hi Marcus,

 

I'm new with Minemeld and never used the old Azure Miner. 

But I found the list we get using the new miner very big and investigated further.

 

The solution I found is to set a filter in the Minemeld processor, selecting only the prefixes having a null value "" into the azure_system_service_list" :

 

Christophe_Savoy_0-1612455781625.png

 

This way you get all the "AzureCloud" prefixes, which should be like the old miner...

 

If that's not enough, you can also play with some "Regional" filters, like this :

 

Christophe_Savoy_1-1612456096905.png

 

 

NB : Do not use this basic filter 

  - azure_region == 'uksouth'

 

Because most prefixes appears twice in the .json file, one in the regional section, and a second time in the 'null' section at the end. And the default Miner retains only the last value it sees, i.e (view of the log) :

Christophe_Savoy_2-1612456351929.png

 

 

View solution in original post

MarcusHil
L1 Bithead

Very cool, I'll give that a try but that looks very much like it would work. Thanks!

MarcusHil
L1 Bithead

Hi Christophe,

Where do you see the output in your third screenshot (with the arrows)? Try as I might I can't seem to find it in the logs...

MarcusHil
L1 Bithead

Nevermind, two seconds after posting I found it  

Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!

The LIVEcommunity thanks you for your participation!