Troubleshooting tools

Reply
L4 Transporter

Troubleshooting tools

I am new to firewalls and new to PA so I really need to find some tools and technique to be able to troubleshoot issues on my PA

L7 Applicator

Re: Troubleshooting tools

Hi

A good start would be here : Education and Training - Palo Alto Networks Courses

There are several interesting courses that could come in handy including a free online e-learning "configuration 101"

hope this helps

Tom

L4 Transporter

Re: Troubleshooting tools

I did the 101 already and it was helpfull and I looked on the online courses but I was hoping to find some free stuff and reference material to always have on hand

L4 Transporter

Re: Troubleshooting tools

So second step is to read this community and learn from us. Beleave me - it's working.

I'm a good example, I didn't pass any PA courses or exams, but after two years I have a good community score and knowledge :smileyhappy:

Regards

SLawek

Not applicable

Re: Troubleshooting tools

Here is a good list of CLI commands to help you out:

General system health

·         show system info –provides the system’s management IP, serial number and code version

·         show system statisticsshows the real time throughput on the device

·         show system software status – shows whether various system processes are running

·         show jobs processed – used to see when commits, downloads, upgrades, etc. are completed

·         show jobs all -show any jobs in progress

·         show job id <id#> -to show any warning/error in configuration

·         clear job id <id#> -to clear a hung job

·         show system disk-space- show percent usage of disk partitions

·         show system logdb-quota – shows the maximum log file sizes

·         debug dataplane internal vif link – show management interface (eth0) counters

·         show system state filter cfg.general.max* - To display the System Limits for objects, profiles, and policies

To monitor CPUs

·         show system resources -  shows processes running in the management plane similar to “top” command

·         show running resource-monitor – used to see the resource utilization in the data plane, such as dataplane CPU utilization

·         less mp-log mp-monitor.log – Every 15 minutes the system runs a script to monitor management plane resource usage, output is stored in this file.

·         less dp-log dp-monitor.log - Every 15 minutes the system runs a script to monitor dataplane resource usage, output is stored in this file.

General dropped packet troubleshooting

·         ping source <IP_addr_src_int> host <IP_addr_host> - allows to ping from the specified FW source interface

·         ping host <IP> - ping from the MGT interface

·         show session all | match – used to show specific sessions in the session table.  You can enter any text after the word match.  A good example would be a source or destination IP or an application

·         show session all | filter destination <IP> dest- shows all sessions going to a particular dest IP and port <port>- port

·         show session all filter type predict To show any pin-hole applications (e.g.FTP)

·         show session id – shows the specifics behind a particular session by entering the ID number after the word “id”

·         show counter interface – shows interface counters

·         show counter global | match drop – used to troubleshoot dropped packets

·         show counter global delta yes | match [source ip|dest ip| drop | error  | frag ] – show counter changes since last time ran this command, filter on particular keyword

·         show counter global filter packet-filter yes delta yes – show counter changes since last time ran this command, filter on debug filter

·         show counter global filter delta yes – show counter changes since last time ran this command

NAT

·         show running nat-policy- shows current NAT policy table

·         show running ippool- use to see if NAT pool leak

·         test nat-policy-match – simulate traffic going through the device, what NAT policy will it match?

Routing

·         show routing route – displays the routing table

·         test routing fib-lookup virtual-router <VR_name> ip <IP_addr_trying_reach> - finds which route in the routing table will be used to reach the IP address that you are testing

Routing Debug Commands

·         debug routing global on debug

·         less mp-log routed.log - To view the log

·         tail follow yes mp-log routed.log - To view the log in real time

Policies

·         show running security-policy – shows the current policy set

·         test security-policy-match from trust to untrust destination <IP>- simulate a packet going through the system, which policy will it match?

PAN Agent

·         show user pan-agent statistics – used to see if the agent is connected and operational.  Status should be connected OK and you should see numbers under users, groups and IPs.

·         show pan-agent user-IDs  - used to see if the FW has pulled groups from the PANAgent

·         show user ip-user-mapping – used to see IP to username mappings on the FW

·         clear user-cache all – clears the user-ID cache

·         debug device-server reset pan-agent <name> - reset the firewall’s connection to the specified agent

URL

·         request url-filtering upgrade brightcloud- If URL does not show up on dynamic updates page run this command

·         test url <url or IP> – used to test the categorization of a URL on the FW

·         tail follow yes mp-log pan_bc_download.log – shows the BrightCloud database update logs

·         request url-filtering download status – shows the status of the database download (essentially the very last line from the pan_bc_download.log file)

·         debug dataplane show url-cache statistics– shows statistics on the URL cache

·         show counter global | match url – shows statistics on URL processing

·         clear url-cache – used to clear the URL cache- cache contains 100k of the most popular URLs on this network

·         show log url direction equal backward- view the URL log, most recent entries first

·         To test connectivity to the BrightCloud servers:

o   ping host service.brightcloud.com

o   ping host database.brightcloud.com

Log viewing / deleting[1]

·         show log [ system | traffic | threat ] direction equal backward – will take you to the end of the specified log

·         show log [ system | traffic | threat ] direction equal forward – will take you to beginning of the specified log

·         clear log [ traffic | threat | acc ] – clear everything in the specified log

·         show log traffic receive_time in ? - pick a timeframe from the list

·         sho log traffic app equal gmail - show only gmail traffic in log

IPSec

·         To view detailed debug information for IPSec tunneling:

1.      debug ike global on debug

2.      less mp-log ikemgr.log

3.      test vpn ike-sa gateway <gw_name> - initiates traffic to bring up tunnel

4.      show vpn ike-sa gateway <gw_name> - to see if phase 1 is up

5.      show vpn ipsec-sa tunnel <tunnel name> - to see if phase 2 is up

6.      show vpn flow – to see all active tunnels

7.      sho vpn flow <name> or tunnel-id <id#>  -to see detailed info on the tunnel

HA

·         show high-availability state – shows the HA state of the FW you are on

·         show high-availability state-synchronization – shows if the FWs are synced

·         show high-availability path-monitoring – shows the status of path monitoring

·         request high-availability state suspend – this will suspend active box and make the current passive device active

·         request high-availability clear-alarm-led – this will clear the HA failover alarm on the unit

Vsys

·         set system setting target-vsys <vsys #> -to enter a vsys

·         set system setting target-vsys none – to exit a vsys

Software, Content, and Licenses

·         To upgrade the software on the FW:

1.      tftp import software from <IP_addr_tftp_server> file <filename>

2.      request system software install file <filename>

3.      request restart system

·         request system software [info | check | download | install ]manipulate PANOS software from the CLI

·         To upgrade the content on the FW:

1.      tftp import content from <IP_addr_tftp_server> file <filename>

2.      request content upgrade install file <filename>

·         request content downgrade install previous –downgrade to the previous content version

·         request system private-data-reset- to clear config and logs/reports

·         debug swm [ status | list | revert ] will show possible code to install, or code that was installed. “revert” is used to revert to last running OS version without having to do a factory reset (such as from 4.0 to back to 3.1)

·         request license info – shows the license installed on the device

·         delete license key ? use to delete a license file if having issues and want to retrieve new licenses, use question mark to list file names, only delete the files you see fit

Config diff/force/cli format

·         show config diff- compares two versions of the config

·         commit force- perform a commit, even if there are errors

·         set cli config-output-format set- use to view the config in “set” format from within the configure prompt (#)

Misc

·         set deviceconfig setting session tcp-reject-non-syn no – used to ignore SYN when creating sessions; confirm command took effect with show session info

·         set deviceconfig setting session offload no –- makes all packets go through CPU, otherwise all fastpath packets just go through EZ chip (turns off session offload to fastpath); confirm command took effect with show session info

·         set deviceconfig setting tcp drop-out-of-wnd <yes|no>; confirm command took effect with show running tcp state

·         debug dataplane pool statistics - this will show the different dataplane buffers and can be used to see if the system is nearing capacity in certain functionality.

·         show system state filter sys.s(x).p(x).phy -command to see physical media

·         set cli pager off - To disable the more function

·         delete network interface ethernet ethernet1/x- deletes any setting on the interface

·         request system private-data-reset- delete private data but keeps software,content installations

·         show system files- to see if FW generated any core-files

·         grep mp-log * pattern (what your searching for-name)- to search all logs for a specific word

·         less dp0-log brdagent.log- to check to see if you have physical errors on interface

·         less dp0-log mprelay.log- to check to see if you have physical errors on interface

·         show system state filter-pretty sw.comm.s1.*.session-info | match active- to see number of sessions on each data-plane

·         https://x.x.x.x/esp/restapi.esp?type=keygen&user=admin&password=admin – To generate a API key

Debug Commands

·         debug dataplane packet-diag show setting - to see if any filters or capture are set

·         debug dataplane packet-diag set filter on - to turn on filter

·         debug dataplane packet-diag set filter match source x.x.x.x destination x.x.x.x destination-port X file test.pcap

·         debug dataplane packet-diag set capture stage <receive,drop,firewall,transmit> file <file name>

·         debug dataplane packet-diag set caprture on - to turn capture on

·         view-pcap follow yes <filter-pcap,debug-pcap> test.pcap yes- this allows you to view the data real time

·         view-pcap filter-pcap <file name>

CLEAN UP COMMANDS:

·         debug dataplane packet-diag set capture off - to stop capturing data

·         debug dataplane packet-diag set filter off- shut off filter

·         delete debug-filter test.pcap - to delete the file

Debug Flow Basic

·         debug dataplane packet-diag filter on

·         debug dataplane packet-diag set filter source x.x.x.x dest y.y.y.y

·         debug dataplane packet-diag set log on

·         Generate traffic

·         less dp0-log pan_packet_diag.log

CLEAN UP COMMANDS:

·         debug dataplane packet-diag clear log log

·         debug dataplane packet-diag filter off

·         debug dataplane packet-diag set log off



[1] Arguments that are shown with square braces and pipe symbol mean that you choose one of the arguments listed. For example, [ arg1 | arg2 | arg3 ] means you select either “arg1” or “arg2” or “arg3”.

L4 Transporter

Re: Troubleshooting tools

Awesome thanks and knowing how to interpret what I find would be really useful.

L7 Applicator

Re: Troubleshooting tools

The troubleshooting collection from Support:

Troubleshooting Palo Alto Networks Hardware Issues

Troubleshooting User-ID: Group and User-to-IP Mapping

Packet Based Troubleshooting - Configuring Packet Captures and Debug Logs

Troubleshooting User Activity Reports

Troubleshooting GlobalProtect, PAN-OS 4.1

How to Troubleshoot VPN Connectivity Issues

Understanding PAN-OS NAT

Steve Puluka BSEET - IP Architect - DQE Communications (Metro Ethernet/ISP)
ACE PanOS 6; ACE PanOS 7; ASE 3.0; PSE 7.0 Foundations & Associate in Platform; Cyber Security; Data Center
Highlighted
L4 Transporter

Re: Troubleshooting tools

Awesome thanks

Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the Live Community as a whole!

The Live Community thanks you for your participation!