Azure Palo Alto - ARP Not Found

[ashleyk]

Hello,

Im having a problem with my PA deployment in Azure where i get ARP Not Found counters increase. I can normally resolve the issue by manually adding an ARP entry to the interface with the MAC of `12:34:56:78:9a:bc` but its really not a solution, rather a workaround.

The architecture is similar to the above. My Azure PointToSite Gateway gives has a client pool of 11.0.0.0/8.
I can peer the spoke network and setup the routing no problem, however unless i explicitly add the spoke VM IP address (10.120.0.10) to the ARP table for ethernet1/2, the traffic wont get there.

From a linux VM connected to the Gateway, i get allocated 11.0.0.130. From this VM i can SSH into the spoke machine at 10.120.0.10 via the PA. Only once ive added the 120.120.0.10 12:34:56:78:9a:bc ARP entry to ethernet1/2 and only once ive added my client address 11.0.0.130 mac 12:34:56:78:9a:bc to ethernet1/3. Otherwise the respective interfaces No ARP counters increase when i run `show interface ethernet1/x'.

This is just a development environment (for now) so ive disabled most NSGs. Ive set my security policy to allow everything.

Why does my connection not work unless i add in the ARP manually? This isnt going to be feasible long term, I cant add an ARP entry for every endpoint in every spoke.

My entire infrastructure is deployed via Terraform.

Looking at this (unresolved) post, it seems that im not the only person having problems - https://live.paloaltonetworks.com/t5/general-topics/incomplete-arp-when-deployed-in-azure/td-p/31533...

Also there somebody else was having ARP problems with Azure, but they answered their own question and it didnt really help.

https://live.paloaltonetworks.com/t5/vm-series-in-the-public-cloud/azure-no-arp/td-p/259088

I dont think the problem is related to my VPN. There is another situation where No ARP appears.

I tried setting up the untrusted subnet access as per https://live.paloaltonetworks.com/t5/vm-series-in-the-public-cloud/second-public-ip-for-vm-300-hoste...

I created an Azure Public IP, added it to the untrusted NIC. I added the private and public (secondary) IPs to the ethernet1/1 configuration. I then setup a NAT rule to translate the traffic to that public IP address.

I have routes on spoke to send all traffic to the PA. On the PA i have a static route which sends traffic to ethernet1/1 by default (0.0.0.0/0). From the spoke VM if i ping 8.8.8.8, i see No Arp counts increating on the ethernet1/1 interface. The NAT rule is getting hit by the looks of it. Its one thing to add a manual ARP entry to the Azure fabric MAC for Azure resources, but i cant manually do that for public internet resources.

Here is the dashboard to show what verison of the PA i am using:

These are the VM Series config values:

```

vm_size = "Standard_D3_v2"
sku = "bundle2"
publisher = "paloaltonetworks"
product = "vmseries1"

```

Deployed in the UK South region.

PANOS Version seems to be 9.1.0

When i run, `show interface all`, all of the interfaces have MAC addresses assigned. They are not the standard `12:34:56:78:9a:bc` address.

Im new to PaloAlto, so im hoping there is something simple im missing here. Im finding it a bit tricky as i thought Azure was meant to handle the layer two stuff.

[ashleyk · Accepted solution]

So i ended up raising a ticket with Palo Alto support and they helped with this (seriously impressed with their support!).

It turns out that other post was a big hint - https://live.paloaltonetworks.com/t5/vm-series-in-the-public-cloud/azure-no-arp/td-p/259088

In the reference architecture guide (https://www.paloaltonetworks.com/resources/guides/azure-architecture-guide), on page 29 it says:

"By default, when a firewall interface obtains a default gateway from DHCP, it installs a default route. To ensure proper
traffic flow, you should modify the firewall configuration so that default routes are static and not obtained through
DHCP. To allow the firewall to reach virtual machines and services within the VNet, set up static routes to the VNet
internal networks on the firewall’s private interface. Even though Azure networking does not use traditional forwarding,
you still configure the route’s next hop as if the network has a default gateway. Azure reserves the first address in the
subnet (example: .1 in a /24) as the subnet’s default router address."

So all i needed to do was update my virtual routes to point to .1 of the interface subnet. In a subnet, Azure reserves the first 5 IP addresses for themselves. It seems that .1 is the gateway address for the subnet.

Regarding the outbound internet access, i was pretty close....
On the reference architecture guide, on page 32 it states:

"For virtual machines behind the firewall to communicate to devices on the internet, the firewall must translate the
source IP address of the outbound traffic to an IP address on the public subnet. Azure then translates the source IP
address again as the outbound traffic leaves the VNet. When you associate a public IP address with an internal IP
address used in the NAT policy, Azure translates the outbound traffic to the public IP address"

So what i did was create a Public IP, associate it to my public NIC. Then on PA, i switched to static IP allocation for the public interface. I manually added 10.110.2.4 and the public IP address. Then on my NAT rules, i added Dynamic Host and Port, changed to interface, selected ethernet1/1 and put the 10.110.2.4 address (not the public address) and i then got the outbound working.

Im seriously impressed with the Palo Alto support. 2/3 hours after i raised the ticket, i was on a zoom call and my problems were resolved.

So the answer was in the documentation all along - who'd have guessed!

[crostron76]

Thanks @ashleyk for sharing this.  Setting up and Azure PA-VM today I made these same initial setup errors.  My deployment is almost identical to yours, minus the VPN.  Once I moved to Static IP address on the public interface and static routes as prescribed by you, traffic started flowing.