topic Re: Azure multiple VM-series with UDR and Load balancers in VM-Series in the Public Cloud

Azure multiple VM-series with UDR and Load balancers

michelvankessel — Wed, 14 Jun 2017 14:44:17 GMT

Hello,

At this moment I am doing a PoC for a client in Azure with two VM-300 in the so called "Sandwich" mode. So for traffic coming from the internet I have the following path: ELB > VM-300 (x2) > ILB > Webserver (x2). Both VM-300 and Webservers are both in a seperate availabilty set.

I managed to load balance the traffic from the internet on the ELB, over both VM-300s and the via the ILB to both Webservers. So far so good!

Now I ask myself the question, how do I load balance the traffic, that initiates from the webservers to the internet? on the UDR I can only point to one VM-300, not both. I cannot use a ILB for anykind of SNAT or put in in routed mode.

As far as I can see, they only thing I can do is change the UDR via a script (e.g. zookeeper) in case one of the VM-300s goes down. how do you guys handle this? How do you achieve load balance/HA for outbound server traffic to the internet or other zones in the case of two VM-300s?

Many thanks

regards

Michel van Kessel

Re: Azure multiple VM-series with UDR and Load balancers

Warby — Wed, 14 Jun 2017 14:58:40 GMT

Hello,

You have two choices here:

One is to deploy another Azure load balancer just for outbound traffic and point your outbound UDR (default route) to the outbound load balancer. Azure allows such a configuration. The problem is, the healthcheck from the load balancer is always from the same IP address. So if you have one load balancer for inbound traffic and one load balancer for outbound traffic pointing to the same set of VM-Series firewalls, the firewalls will receive healthchecks from two different load balancers on two different interfaces but coming from the same IP address. This can be solved using two virtual routers on the firewall - one for each interface.

The other option is to split inbound and outbound traffic to two different set of firewalls. Then each set of firewalls will only receive healthchecks from one load balancer. This adds VM-Series cost to the solution but it simplifies the routing and also has the advantage of having policy separation (inbound versus outbound). Also, the outbound firewalls can be used for multiple applications/resource groups in a hub and spoke model

HTH,

Warby

Re: Azure multiple VM-series with UDR and Load balancers

michelvankessel — Wed, 14 Jun 2017 20:28:23 GMT

Hello Warby

Thanks for the quick response! I am going to try the multiple router approuch and will return with the outcome

Quick question. Why doesn't it work with one virtual router? I mean, from a deeper technical point of view.

Regards

Michel van Kessel

Re: Azure multiple VM-series with UDR and Load balancers

Warby — Wed, 14 Jun 2017 20:40:37 GMT

It's a basic routing problem. Each subnet (even a /32) must be maped in the route table to/from a unique interface. The Azure load balancer always sources the probe from 168.63.129.16. If we have a default route for the external interface (say ethernet1/1), all responses to the healthcheck will go out ethernet1/1 even if the healthcheck came from the internal load balancer (on say ethernet1/2). This results in an asymmetric path and the health probe fails.

If we then created a specific 168.63.129.16/32 route to use ethernet1/2, then the probes from the external load balancer would become assymetric and those healthchecks would fail.

With multiple route tables, we could have one route table with a 168.63.129.16/32 static route that points to ethernet1/1 and another route table with a static entry for the same 168.63.129.16/32 for ethernet1/2.

Make sense?

Warby

Re: Azure multiple VM-series with UDR and Load balancers

michelvankessel — Thu, 15 Jun 2017 12:06:03 GMT

Hello Warby.. I was completely looking into the wrong direction! Sometimes there is a simple solutions and it makes complete sense!

I've got it all running now.

One question. do we need Floating IP (direct server return) enabled on the rules for the outbound LB?

Thank you

Michel van Kessel

Re: Azure multiple VM-series with UDR and Load balancers

kblackstone — Mon, 23 Oct 2017 17:34:41 GMT

Yes, enable floating ip for your outbound rules so the destination remains true as requested.

Re: Azure multiple VM-series with UDR and Load balancers

junior_r — Mon, 11 Dec 2017 01:29:24 GMT

Hi,

For the outbound LB what was the prober you setup?

Thanks

Re: Azure multiple VM-series with UDR and Load balancers

junior_r — Mon, 11 Dec 2017 19:36:38 GMT

Hi,

Anyone can let me know prober for outbound traffic?

Thanks

Re: Azure multiple VM-series with UDR and Load balancers

kblackstone — Tue, 12 Dec 2017 01:10:02 GMT

HI junior_r,

There are a few options when it comes to setting up the outbound probe but the most common that I have seen is to probe to port 22 on the firewall and enable the related interface management profile (for ssh) on the interface that will receive the traffic first. You may need to add the route to support return traffic to the probe source ip address: 168.63.129.16.

Alternitavely, you can probe to port 80 or 443 and write a NAT rule that sends traffic from 168.63.129.16 on the appropriate zone out to 8.8.8.8 on the Internet, or some other website that you wish to verify connectivity against. Leverage destination NAT to accomplish this.

-kb

Re: Azure multiple VM-series with UDR and Load balancers

junior_r — Tue, 12 Dec 2017 01:16:16 GMT

Thanks

Quick Question with this setup how does the LB know to keep the traffic symmetric for TCP FLOW.

Web Server -> LB -> PAN FW1 / PAN FW2 -> Internet

Thanks

Re: Azure multiple VM-series with UDR and Load balancers

kblackstone — Tue, 12 Dec 2017 01:20:24 GMT

Each Load Balancer rule has session affinity settings: none, client ip, client ip and protocol, which keeps the sessions going to the same back-end resource and symmetric.

This link might shed some light: https://docs.microsoft.com/en-us/azure/load-balancer/load-balancer-distribution-mode

-kb

Re: Azure multiple VM-series with UDR and Load balancers

junior_r — Tue, 12 Dec 2017 01:23:09 GMT

@kblackstone wrote:
Each Load Balancer rule has session affinity settings: none, client ip, client ip and protocol, which keeps the sessions going to the same back-end resource and symmetric.

This link might shed some light: https://docs.microsoft.com/en-us/azure/load-balancer/load-balancer-distribution-mode

-kb

this is like the F5 then Thanks again

Re: Azure multiple VM-series with UDR and Load balancers

junior_r — Wed, 13 Dec 2017 02:26:47 GMT

Hi,

Since PINGS are not aloud on Azure LB how are you guys working with applications that require ping?

Server 1 -> LB -> PAN -> DB

Thanks

Re: Azure multiple VM-series with UDR and Load balancers

nsbradley88 — Mon, 08 Jan 2018 17:06:14 GMT

@junior_r Which endpoint is supposed to receive the ping from the application? The LB will pass the ping along like it would any other traffic. Is the ping destined for the LB from a standpoint of a healthcheck?

You should be able to resolve a response from the LB on 443. Give PsPing a shot and see for yourself. Not sure how flexible your application is from a ping perspective.

Re: Azure multiple VM-series with UDR and Load balancers

BalaBWV — Wed, 16 Sep 2020 13:13:03 GMT

Is this working with multiple VR, I want to implement the same

Re: Azure multiple VM-series with UDR and Load balancers

JimMcGrady — Fri, 17 Dec 2021 06:52:59 GMT

Ive found the symmetric flow to be an issue. I have an external client coming in through an Azure public load balancer, through my pair of PA firewalls, and into an internal load balancer which delivers the traffic to the servers. However the reply traffic doesnt work unless i make the firewalls source NAT the inbound traffic. The default route outbound from the backend servers is an internal load balancer across the two firewalls. sometimes the reply traffic goes back through the other firewall, therefore breaking symmetry.