topic Re: arp not found in General Topics

arp not found

AndrewMoore — Thu, 23 Jun 2016 13:35:06 GMT

Yesterday I attempted to move our Internet connection from a copper interface on ethernet1/1 to fiber optic on ethernet 1/13 on a Palo Alto 3020.

I ensured both interfaces were members of the same security zone and modified the Default route of default-vr to use ethernet 1/13 instead of 1/1.

While I and the NOC technician confirmed the fiber port was configured and up, I could not ping out to the Internet.

During that time ran a show interfaces ethernet1/13 and found 13 instances of "arp not found" under the Logical interface counters read from CPU.

It seems that maybe I needed to clear the arp table? I say so because I believe the firewall was trying to reach the MAC of our next hop ( our fiber switch ) through ethernet1/1 instead of ethernet1/13.

Any advice would be greatly appreciated!

Re: arp not found

reaper — Thu, 23 Jun 2016 13:44:46 GMT

was the next-hop IP identical for the fiber as what it was on copper ?

you could try clearing the arp cache with

> clear arp ethernet1/1

to verify if the issue's related to arp, you can check the global counters for more detailed information:

> show counter global filter delta yes | match arp

Re: arp not found

Roby_Sreejith — Thu, 23 Jun 2016 13:49:42 GMT

Have you changed the zones? like ethernet 1/1 zone name and ethernet 1/13 zone?

If same zone you are using, no issue.

otherwie security rule should be created

Re: arp not found

AndrewMoore — Thu, 23 Jun 2016 14:01:57 GMT

Thanks for the reply @Roby_Sreejith. I confirmed before the switch over that the new fiber port ethernet1/13 had been assigned to the same seucurity zone, L3-Untrusted, as ethernet1/1 had been assigned to.

Re: arp not found

AndrewMoore — Thu, 23 Jun 2016 14:06:00 GMT

Thanks @reaper. We have a Cisco 3400 fiber switch that is our next hop from the 3020. Within the static-route tab of the default-vr I did not change the next hop.

Since the fiber switch remains as our next hop I don't believe anything is out of place with regard to next hop configuration.

Appreciate the suggestions about clearing arp cache and checking arp stats!

Re: arp not found

AndrewMoore — Thu, 23 Jun 2016 14:11:10 GMT

This was the state of our fiber port yesterday before I abandoned ship and moved back over to copper on eth1/1

bob@pafw> show interface ethernet1/13
--------------------------------------------------------------------------------
Name: ethernet1/13, ID: 28
Link status:
Runtime link speed/duplex/state: 1000/full/up
Configured link speed/duplex/state: 1000/auto/auto
MAC address:
Port MAC address d4:f4:be:ab:cd:ef
Operation mode: layer3
Untagged sub-interface support: no
--------------------------------------------------------------------------------
Name: ethernet1/13, ID: 28
Operation mode: layer3
Virtual router default-vr
Interface MTU 1500
Interface IP address: 17.22.113.34/28
17.22.113.37/32
Interface management profile: N/A
Service configured: IKE
Zone: L3-Untrusted, virtual system: vsys1
Adjust TCP MSS: no
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
Physical port counters read from MAC:
--------------------------------------------------------------------------------
rx-broadcast 0
rx-bytes 0
rx-multicast 0
rx-unicast 0
tx-broadcast 8
tx-bytes 512
tx-multicast 0
tx-unicast 0
--------------------------------------------------------------------------------
Hardware interface counters read from CPU:
--------------------------------------------------------------------------------
bytes received 1614
bytes transmitted 336
packets received 13
packets transmitted 8
receive incoming errors 0
receive discarded 0
receive errors 0
packets dropped 0
--------------------------------------------------------------------------------
Logical interface counters read from CPU:
--------------------------------------------------------------------------------
bytes received 1614
bytes transmitted 336
packets received 13
packets transmitted 8
receive errors 0
packets dropped 0
packets dropped by flow state check 0
forwarding errors 0
no route 0
arp not found 13
neighbor not found 0
neighbor info pending 0
mac not found 0
packets routed to different zone 0
land attacks 0
ping-of-death attacks 0
teardrop attacks 0
ip spoof attacks 0
mac spoof attacks 0
ICMP fragment 0
layer2 encapsulated packets 0
layer2 decapsulated packets 0
--------------------------------------------------------------------------------

bob@pafw>

Re: arp not found

TranceforLife — Thu, 23 Jun 2016 14:23:37 GMT

Hi Andrew.

So both ports are in the same subnet? Any logs from the Cisco switch? ARP table

Re: arp not found

AndrewMoore — Thu, 23 Jun 2016 14:39:48 GMT

Hey @TranceforLife,

Our ISP has assigned us a block of /28 IP addresses. In my description I was trying to emphasize that I copied/mirrored the same settings on eth1/13 from eth1/1. As a result both interfaces have the same addresses assigned, just not at the same time.

In other words, before pulling the copper line and installing the fiber I did the following to produce a valid configuration that would commit. After the following steps I pull copper and plug in optical.

1) Assign eth1/13 to same Security Zone as eth1/1 - that is L3-Untrusted.

2) Remove IP addresses AAA.BBB.CCC.34/28 & AAA.BBB.CCC.37 and assign to eth1/13

3) Modify default-vr to include int1/13 and assign as interface to be used for the Default Route.

4) Move IKE Gatway interface assignment from eth1/1 to eth1/13

I do not own, manage, or have console access to the Cisco switch but could inquire with the NOC for any specific statistics if you have any suggestions.

Much appreciated.

Re: arp not found

TranceforLife — Thu, 23 Jun 2016 14:55:49 GMT

Hi Andrew,

Try to run PCAP on the 1/13 interface to see for the ARP packets. Configure the filter for this interface and capture Receive and Transmit packets only. KB article below:

https://live.paloaltonetworks.com/t5/Learning-Articles/How-to-Run-a-Packet-Capture/ta-p/62390

Try GUI option, it is easier

Cheers

Re: arp not found

Roby_Sreejith — Thu, 23 Jun 2016 14:55:45 GMT

Have you verified the NAT policy?

Just make sure you have migrated everything belongs to e1/1 to e1/13 in NAT

Re: arp not found

AndrewMoore — Thu, 23 Jun 2016 15:16:45 GMT

Excellent suggestion @Roby_Sreejith!

In my previous posts it's clear that I did not modify the Dynamic NAT policy to use eth1/13 instead of eth1/1.

However, I was trying to ping 8.8.8.8 from the Palo Alto CLI using the AAA.BBB.CCC.34/28 and AAA.BBB.CCC.37 IP addresses, which, I suspect, do not get translated using NAT.

Hope my assumptions are right but happy to be wrong and learn something new (-:

I'll be switching over again today around 12:15 CST so I'll be sure to try your and other suggestions, like clearing ARP tables, capturing packets and filtering ARP traffic, etc.

Thank you again!

Re: arp not found

TranceforLife — Thu, 23 Jun 2016 15:34:00 GMT

Agree with Roby.

Please confirm Cisco port MAC address and that you actually can see it in your arp table:

> show arp ethernet1/13

Also, when pinging 8.8.8.8 try to source the ping from the fibre interface (ethernet1/13)

> ping source (fibre interface IP) host 8.8.8.8

Cheers

Re: arp not found

AndrewMoore — Thu, 23 Jun 2016 16:25:07 GMT

Thanks again @TranceforLife. I can confirm that during the switch to fiber on ethernet1/13 I ran pings from the PA's eth1/13 interface like so:

> ping source AAA.BBB.CCC.34 host 8.8.8.8

I will confirm the MAC address tied to the current interface (Cisco) connected to eth1/1.

Thank you to everyone for the big assist this morning.

Re: arp not found

AndrewMoore — Thu, 23 Jun 2016 19:08:48 GMT

Thank you to everyone for your suggestions.

Wanted to let you know the conversion from copper to fiber was a success over lunch today

The solution was I had to issue gratuitous ARPs from a few IPs in my address block - this quickly solved everything.

Cheers to everyone!

Re: arp not found

TranceforLife — Thu, 23 Jun 2016 22:25:00 GMT

Hi Andrew,

Glad to hear it! You can mark this topic as "fixed" now

All the best.