Azure multiple VM-series with UDR and Load balancers

[michelvankessel]

Hello,

At this moment I am doing a PoC for a client in Azure with two VM-300 in the so called "Sandwich" mode. So for traffic coming from the internet I have the following path: ELB > VM-300 (x2) > ILB > Webserver (x2). Both VM-300 and Webservers are both in a seperate availabilty set. 

I managed to load balance the traffic from the internet on the ELB, over both VM-300s and the via the ILB to both Webservers. So far so good!

Now I ask myself the question, how do I load balance the traffic, that initiates from the webservers to the internet? on the UDR I can only point to one VM-300, not both. I cannot use a ILB for anykind of SNAT or put in in routed mode.

As far as I can see, they only thing I can do is change the UDR via a script (e.g. zookeeper) in case one of the VM-300s goes down. how do you guys handle this? How do you achieve load balance/HA for outbound server traffic to the internet or other zones in the case of two VM-300s?

Many thanks

regards

Michel van Kessel

[Warby · Accepted solution]

Hello,

You have two choices here:

One is to deploy another Azure load balancer just for outbound traffic and point your outbound UDR (default route) to the outbound load balancer.  Azure allows such a configuration.  The problem is, the healthcheck from the load balancer is always from the same IP address.  So if you have one load balancer for inbound traffic and one load balancer for outbound traffic pointing to the same set of VM-Series firewalls, the firewalls will receive healthchecks from two different load balancers on two different interfaces but coming from the same IP address.  This can be solved using two virtual routers on the firewall - one for each interface.

The other option is to split inbound and outbound traffic to two different set of firewalls.  Then each set of firewalls will only receive healthchecks from one load balancer.  This adds VM-Series cost to the solution but it simplifies the routing and also has the advantage of having policy separation (inbound versus outbound).  Also, the outbound firewalls can be used for multiple applications/resource groups in a hub and spoke model

HTH,

Warby

[michelvankessel · Accepted solution]

Hello Warby

Thanks for the quick response! I am going to try the multiple router approuch and will return with the outcome

Quick question. Why doesn't it work with one virtual router? I mean, from a deeper technical point of view. 

Regards

Michel van Kessel

[Warby]

It's a basic routing problem.  Each subnet (even a /32) must be maped in the route table to/from a unique interface.  The Azure load balancer always sources the probe from 168.63.129.16.  If we have a default route for the external interface (say ethernet1/1), all responses to the healthcheck will go out ethernet1/1 even if the healthcheck came from the internal load balancer (on say ethernet1/2).  This results in an asymmetric path and the health probe fails.

If we then created a specific 168.63.129.16/32 route to use ethernet1/2, then the probes from the external load balancer would become assymetric and those healthchecks would fail.

With multiple route tables, we could have one route table with a 168.63.129.16/32 static route that points to ethernet1/1 and another route table with a static entry for the same 168.63.129.16/32 for ethernet1/2.

Make sense?

Warby

[michelvankessel]

Hello Warby.. I was completely looking into the wrong direction! Sometimes there is a simple solutions and it makes complete sense!

I've got it all running now.

One question. do we need Floating IP (direct server return) enabled on the rules for the outbound LB?

Thank you

Michel van Kessel

[kblackstone]

Yes, enable floating ip for your outbound rules so the destination remains true as requested.

[junior_r]

Hi,

For the outbound LB what was the prober you setup?

Thanks

[junior_r]

Hi,

Anyone can let me know prober for outbound traffic?

Thanks

[kblackstone]

HI junior_r,

There are a few options when it comes to setting up the outbound probe but the most common that I have seen is to probe to port 22 on the firewall and enable the related interface management profile (for ssh) on the interface that will receive the traffic first. You may need to add the route to support return traffic to the probe source ip address: 168.63.129.16.

Alternitavely, you can probe to port 80 or 443 and write a NAT rule that sends traffic from 168.63.129.16 on the appropriate zone out to 8.8.8.8 on the Internet, or some other website that you wish to verify connectivity against. Leverage destination NAT to accomplish this.

-kb

[junior_r]

Thanks

Quick Question with this setup how does the LB know to keep the traffic symmetric for TCP FLOW. 

Web Server -> LB -> PAN FW1 / PAN FW2 -> Internet

Thanks

[kblackstone]

Each Load Balancer rule has session affinity settings: none, client ip, client ip and protocol, which keeps the sessions going to the same back-end resource and symmetric.

This link might shed some light: https://docs.microsoft.com/en-us/azure/load-balancer/load-balancer-distribution-mode

-kb

[junior_r]

---

@kblackstone wrote:

Each Load Balancer rule has session affinity settings: none, client ip, client ip and protocol, which keeps the sessions going to the same back-end resource and symmetric.

This link might shed some light: https://docs.microsoft.com/en-us/azure/load-balancer/load-balancer-distribution-mode

-kb

---

this is like the F5 then Thanks again

[junior_r]

Hi,

Since PINGS are not aloud on Azure LB how are you guys working with applications that require ping?

Server 1 -> LB -> PAN -> DB

Thanks

---

[nsbradley88]

@junior_r Which endpoint is supposed to receive the ping from the application? The LB will pass the ping along like it would any other traffic. Is the ping destined for the LB from a standpoint of a healthcheck? 

You should be able to resolve a response from the LB on 443. Give PsPing a shot and see for yourself. Not sure how flexible your application is from a ping perspective. 

[BalaBWV]

Is this working with multiple VR, I want to implement the same