interesting PPPoE Problem

[vertical]

Hi guys,

i am currently tasked to replace two firewalls we have in the company. The first is a small cisco ASA 5505 for client breakout and a MS TMG(yeah i hate,too) for publishing the Servers.

For the first step I am trying to replace the ASA. WAN connection is established via ADSL and PPPoE. The session is build just fine, traffic is allowed via accessrules and nat'ed correctly, but I just don't can't ping 8.8.8.8 or get any other traffic responses.

Something I've noticed so far is that our ASA always gets an IP from the 84.x.x.x - 86.x.x.x public ip range. The PA received ip like 188.110.x.x or 92.75.x.x. After the PA established the PPPoE Session i checked if it received the default route via CLI and it was there, NAT is source NAT+PAT of course. All interfaces are added in the virtual router and the correct access list has been hit, so i am quite clueless why it is so difficult to get the wan connection working. The other frustrating point is that i have very limited time windows for testing.

I'm grateful for ideas to solve this issue

Here are some screens from the log.

.

[panos]

you have static public ip address ?

I think yes.and you said asa gets public ip different then ppoe paloalto ?

is that correct ?

Try to change authentication setting on paloalto ppoe from auto to other which your isp works with and see if they got the correct ip after commit

[vertical]

- changed the PA authentication from auto to pap which the ASA uses prior to todays testing -> didn't help

- it is a dynamic ip, but the pool doesn't change that much

- yes it is correct the PAN receives an entirely different public ip than the public ips the asa gets assigned.

[panos]

can you ping your default gateway from the ip given using ping source givenip host gateway ?

[mbutt]

just to clarify you are able to browse the traffic however you are not able to ping 8.8.8.8. From the logs it seems you are able to do web-browsing and google analytic. The traffic is being seen in both directions. However when you are pinging you are not seeing the traffic.

Try ping from CLI using source as external ip to make sure you have connection upstream. The command would be

ping source (188.110.47.216 or the ip you are getting on the public interface) host 8.8.8.8

If this is successful then atleast you have connection from public interface to upstream.

Then try to use the internal interface using the same command.

If that fails you can trouble shoot it doing the following

1. Need to setup the filters for the traffic we are interested in. To do this, execute the following steps:

Navigate to Monitor--Packet Capture

Click 'Manage Filters'

Set Filter ID 1 to be the source IP and destination IP of traffic you feel is affected ( leave all other fields blank )

Set Filter ID 2 to be the exact inverse of what you did in step 3 (destination IP in source field, Source IP in destination field)

2. Setup up the captures

Create and name the file stage for a packet capture on all the stages (receive, transmit, firewall and drop)

3. Enable filters and captures 

debug dataplane packet-diag set filter on

debug dataplane packet-diag set capture on

4. open 2 CLI windows

on 1 run the following command to look at the counter ( make sure it run this command once before running the traffic)

show counter global filter packet-filter yes delta yes

on the 2nd window run the following command to look at he sessions

show session all filter source <ip address> destination <ip address>

After your test has been done stop all the captures and filters and see if global counter show you anything why it is dropping the traffic or if you have getting pcap with drop stage.

This will help you narrow down the issue.

Let us know if this helps you resolve the issue.

Thanks

Numan

[vertical]

Hi guys,

so I just finished the Test for my morning timeframe. The results are kinda disappointing and here is the summary:

- pinging from given IP to GW IP didn't work, changed auth prot to chap, pap and auto with no difference

- tested with disabled automatically create default route pointing to peer and also with enabled setting -> no difference

- pinged the given IP from my smartphone -> ping successfully even with no access rule allowing the ping and no mgmt profile set on the IF -> this makes me believe, i've got the wrong account for dialing in

- As Numan advised I've created the packet captures, nothing dropped. Flows and sessions have been created and no drop.pcap was made, only receive, firewall and transmit

- as clarification, no I don't have a working WAN connection with the PAN. The pictures should illustrate, that NAT, routing and access rules are working as planned.

So since everything is looking fine from the firewall angle I guess I have to get back to my boss, to get the correct information for dialing in or do you have any other conclusion?

[panos]

Hi,

if you are not sure about the information for dialing checking this will be good for that time.

that different public ip does not make any sense except that.

[vertical]

looks like the dial in account information i received is correct. gonna test it tomorrow via laptop...getting frustrating learning PAN

[mbutt]

HI Vertical,

I am not sure what software version are you on but i know of an issue not where PA did not support relay session id and that was enhanced in 5.0.3.

While doing the setup you can also tail the pppoed process to see what messages you are seeing

tail follow yes mp-log pppoed.log

Here is the bug which you can also verify in the release notes of 5.0.3

42147—PPPoE sessions were failing on the firewall when a PPPoE relay device was used between the firewall and the PPPoE sever. Issue due to a problem where the firewall was not parsing relay session IDs that begin with Null. As of this release (5.0.3), PAN-OS will now work with PPPoE relay.

Hope this helps in narrowing down the issue.

Thank you

Numan

[vertical]

Hi Numan,

currently it is running on 5.0.6 since the support told me for another case, this would be the most stable and bugfree available.

Here is the output from the log:

Aug 13 10:36:51 Error: pan_pppoed_dp_connect(pan_pppoe_thread.c:261): pan_pppoed_setup_socket() failed

Aug 13 10:36:51 Warning: pan_pppoe_setup_dp_conn(pan_pppoe_thread.c:344): Failed to connect to DP

Aug 14 07:18:47 Error: pan_pppoe_recv_pkt(pan_pppoe_fsm.c:1321): [0x9fdb550/26][0] Drop PPP Sess pkt, sess_id received:1505 does not match pppoe_fsm sess id:0

Aug 14 07:18:47 Error: pan_pppoe_recv_pkt(pan_pppoe_fsm.c:1322):

Aug 14 07:18:48 Error: pan_pppoe_recv_pkt(pan_pppoe_fsm.c:1321): [0x9fdb550/26][0] Drop PPP Sess pkt, sess_id received:1505 does not match pppoe_fsm sess id:0

Aug 14 07:18:48 Error: pan_pppoe_recv_pkt(pan_pppoe_fsm.c:1322):

Aug 14 07:18:50 Error: pan_pppoe_recv_pkt(pan_pppoe_fsm.c:1321): [0x9fdb550/26][0] Drop PPP Sess pkt, sess_id received:1505 does not match pppoe_fsm sess id:0

Aug 14 07:18:50 Error: pan_pppoe_recv_pkt(pan_pppoe_fsm.c:1322):

Aug 14 07:18:51 Error: pan_pppoe_fsm_run(pan_pppoe_fsm.c:1202): [0x9fdb550/26][0] Drop PPPoE PADT pkt, sess_id received:1505 does not match pppoe_fsm sess id:0

Aug 14 07:18:51 Error: pan_pppoe_fsm_run(pan_pppoe_fsm.c:1203):

What is isn't as expected, that the timestamp is from yesterday morning. I could dial in, but had no real outgoing connection later on and also today in the timerange from 7:15 - 8:00.

Also tried to dial in via my laptop and also experienced the same problem. I received an IP from an unsual Pool and had no connection to the ISP or beyond. I've checked on the ASA the little bit of information you get out of the config and the username is correct, but maybe the password was changed and no one in the company can remember so. I always thought, that if you try to dial in with wrong credentials, that you receive an error or so instead of receiving an IP which doesn't work correctly.

[panos]

better talk with isp seems to be wrong password

[mbutt]

Hi Vertical,

Like panos said you might want to talk to ISP. From the logs it seems like the pppoe connection is not being setup and everything is being dropped. Once the password and user name is confirmed than we will be able to move forward. Also in the mean time if you think you can not verify password with ISP then seutp a connection on laptop and see if the existing password and connection works. If it works then there might be something wrong with the firewall and you might need to open a support case. But make sure you validate the user name and password by setting up it on a laptop.

Hope this helps narrow down the issue.

Thanks

Numan

[vertical · Accepted solution]

Hi Guys,

i talked with the ISP and had them reset the password. This was the root cause as it now works as expected.