topic Re: PA-7000 Not passing syslog traffic to Tufin in General Topics

PA-7000 Not passing syslog traffic to Tufin

netzwerk-admin — Thu, 28 Dec 2017 13:10:12 GMT

Hi All,

We have a PA-7000 (7.1) and Tufin (for syslog).

The system was previously setup to forward syslog traffic to Tufin.

Then all of a sudden, Tufin wasn't receiving any traffic.

What I have done so far:

Went through the saved configurations to see when the syslog config was changed.
- From the saved configs, I could not see anything that was changed that affected syslog forwarding.
- No Palo Alto or Tufin updates were installed.
Rechecked the syslog forwarding configuration (at least 5 times as of this writing).
Ran tcpdump on Tufin server
- traffic was not getting to Tufin
- 14:00:54.560060 IP (tos 0x0, ttl 60, id 62901, offset 0, flags [DF], proto UDP (17), length 358)
  10.63.249.5.43067 > tufina01.syslog: [udp sum ok] SYSLOG, length: 330
  Facility user (1), Severity error (3)
  Msg: Dec 28 14:02:06 fw-f-wm-dc-1c.infra.dvag.com 1,2017/12/28 14:02:06,010108000926,SYSTEM,userid,0,2017/12/28 14:02:06,,connect-agent-failure,,0,0,general,high,"TS-Agent Citrix wpsxaaabn02.id(vsys1): Error: Failed to connect to wpsxaaabn02.id(10.61.85.151):5009 details: none",827871,0x0,0,0,0,0,,fw-f-wm-dc-1c
- As seen above, only system type information is reaching Tufin
Ran tcpdump on PA-7000
- 12:14:03.004652 IP 10.63.249.5.53918 > 10.63.98.59.syslog: SYSLOG user.info, length: 226
  12:14:20.101956 IP 10.63.249.5.35845 > 10.63.98.59.syslog: SYSLOG user.info, length: 227
  12:14:31.557722 IP 10.63.249.5.37782 > 10.63.98.59.syslog: SYSLOG user.info, length: 344
  12:14:31.573796 IP 10.63.249.5.53918 > 10.63.98.59.syslog: SYSLOG user.info, length: 253
  12:14:31.640424 IP 10.63.249.5.35845 > 10.63.98.59.syslog: SYSLOG user.info, length: 242
  12:14:32.604810 IP 10.63.249.5.37782 > 10.63.98.59.syslog: SYSLOG user.info, length: 344
  12:14:32.616503 IP 10.63.249.5.53918 > 10.63.98.59.syslog: SYSLOG user.info, length: 253
  12:14:32.682839 IP 10.63.249.5.35845 > 10.63.98.59.syslog: SYSLOG user.info, length: 242
- Traffic Monitoring shows that syslog (udp 514) packets are allowed, Session End Reason 'aged-out'

If someone could push me in the right direction to correct this I would greatly appreciate it.

Regards,

Jasper Freeman

Re: PA-7000 Not passing syslog traffic to Tufin

BPry — Thu, 28 Dec 2017 15:11:25 GMT

@netzwerk-admin,

Just out of curiosity have you attempted to restart the management plane since you began experiancing these issues?

Re: PA-7000 Not passing syslog traffic to Tufin

netzwerk-admin — Thu, 28 Dec 2017 15:14:10 GMT

No we haven't.

I'll give it a try next week. Don't want to make any changes on a Friday. Especially before a long weekend.

Jasper

Re: PA-7000 Not passing syslog traffic to Tufin

BPry — Thu, 28 Dec 2017 15:18:11 GMT

@netzwerk-admin,

If it doesn't work then let us know, but I would assume that this should get things working correctly again.

Re: PA-7000 Not passing syslog traffic to Tufin

netzwerk-admin — Thu, 28 Dec 2017 15:23:02 GMT

Well, scratch that.

A colleague said the system was restarted 8 days ago because a security bug.

So, that answer is yes, the management plane was restarted.

Re: PA-7000 Not passing syslog traffic to Tufin

BPry — Thu, 28 Dec 2017 15:30:28 GMT

@netzwerk-admin,

So that would indicate that this issue is at least more than 8 days old, and didn't start with the restart?

A couple things that I would check.

1) Verify that nobody removed the log-forwarding profile from your security policies. I've seen this happen in the pass with multiple firewall admins.

2) Verify that you can actually get a response from Tufin and that there isn't a routing issue. I would expect the Session End Reason to show as 'aged-out' on Syslog traffic, as the firewall never gets anything to tell it to close the session.

Is Tufin functioning for other devices okay?

Re: PA-7000 Not passing syslog traffic to Tufin

netzwerk-admin — Fri, 29 Dec 2017 06:58:55 GMT

Yes, Tufin is functioning for other devices.

I just restarted the monitoring for the Palo Alto on Tufin and now I'm seeing that syslog traffic is arriving at the Tufin interface.

But, I'm a little confused. If I execute tcpdump it says it is doing the dump an eth0

e13itfd@fw-f-wm-dc-1c(active)> tcpdump filter "src 10.63.249.5 and port 514" snaplen 0
Press Ctrl-C to stop capturing

tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes

Results:

             e13itfd@fw-f-wm-dc-1c(active)> view-pcap mgmt-pcap mgmt.pcap
             07:33:17.762228 IP 10.63.249.5.43067 > 10.63.98.59.syslog: SYSLOG user.error, length: 330
             07:34:15.498597 IP 10.63.249.5.46508 > 10.63.98.59.syslog: SYSLOG user.error, length: 316
             07:35:40.596228 IP 10.63.249.5.38662 > 10.63.98.59.syslog: SYSLOG user.error, length: 316
             07:37:00.204343 IP 10.63.249.5.43067 > 10.63.98.59.syslog: SYSLOG user.error, length: 330
             07:37:35.926424 IP 10.63.249.5.46508 > 10.63.98.59.syslog: SYSLOG user.error, length: 330
             07:37:35.926518 IP 10.63.249.5.38662 > 10.63.98.59.syslog: SYSLOG user.error, length: 329
             07:37:40.205790 IP 10.63.249.5.43067 > 10.63.98.59.syslog: SYSLOG user.error, length: 330
             07:39:09.685971 IP 10.63.249.5.37782 > 10.63.98.59.syslog: SYSLOG user.info, length: 265
             07:39:09.685980 IP 10.63.249.5.35845 > 10.63.98.59.syslog: SYSLOG user.info, length: 277

What confuses me is that on the Palo Alto I don't see any TRAFFIC labled packets. On Tufin a tcpdump with the src=10.63.249.5 also shows no packets at all.

Then I decided to see if there are syslog traffic being sent on the Log Card IP. Tufin is seeing syslog TRAFFIC from the Log Card IP; however, the Palo Alto shows no TRAFFIC at all.

This is confusing.

@BPry, thanks for the help.

Re: PA-7000 Not passing syslog traffic to Tufin

BPry — Fri, 29 Dec 2017 12:54:06 GMT

@netzwerk-admin,

By log card do you mean the SMC?

Re: PA-7000 Not passing syslog traffic to Tufin

netzwerk-admin — Fri, 29 Dec 2017 12:59:03 GMT

@BPry

Actually, it's one of the interface on the NPC-20GQ module.

For example, we have ethernet1/3 (type: Log Card) configured for passing log information to Tufin.