Pa-2020 restart unexpected

[Oskare_o]

Hello

My Pa-2020 restart unexpectedly with no reason aparently. I'm attaching the log so any help me is welcome.

Thanks!

Log:

2014/10/30 11:04:56 info     general        general 0  Connection to Update server: updates.paloaltonetworks.com completed successfu

lly, initiated by 192.168.48.183

!----------- System Restart at 11:59

2014/10/30 11:59:24 info     general        general 0  Management server started. Running version 5.0.9

2014/10/30 11:59:24 info     general        general 0  VPN Disable mode = off

2014/10/30 11:59:26 info     ntpd           restart 0  NTP restart synchronization performed

2014/10/30 11:59:26 high     general        system- 1  The system is starting up.

2014/10/30 11:59:26 info     satd           satd-da 0  SATD daemon is initializing.

2014/10/30 11:59:26 info     vpn            keymgr- 0  KEYMGR daemon is initializing.

2014/10/30 11:59:26 info     ras            rasmgr- 0  RASMGR daemon is initializing.

2014/10/30 11:59:26 info     url-fil        url-eng 0  BrightCloud engine started.

2014/10/30 11:59:26 info     vpn            ike-dae 0  IKE daemon is initializing.

2014/10/30 11:59:26 info     vpn            keymgr- 0  KEYMGR sync all IPSec SA to IKE daemon started.

2014/10/30 11:59:26 info     routing        routed- 0  Route daemon is initializing.

2014/10/30 11:59:26 info     vpn            keymgr- 0  KEYMGR daemon is ready.

2014/10/30 11:59:26 info     satd           satd-da 0  SATD daemon is ready.

2014/10/30 11:59:26 info     vpn            keymgr- 0  KEYMGR sync all IPSec SA to IKE daemon exit.

2014/10/30 11:59:26 info     ras            rasmgr- 0  RASMGR daemon is ready.

2014/10/30 11:59:26 info     vpn            ike-dae 0  IKE daemon is ready.

2014/10/30 11:59:26 info     sslmgr         sslmgr- 0  SSLMGR daemon is ready.

2014/10/30 11:59:26 info     routing        routed- 0  Route daemon is ready.

2014/10/30 11:59:26 info     vpn            keymgr- 0  KEYMGR sync all IPSec SA to Flow no longer needed.

[hshah]

Please provide us output for "show system files".

and show system info.

[HULK]

Hello Oskare_o,

I hope you have attached PAN SYSTEM logs here. Is there any log entries just before the incident happened. (11:59) ?

Thanks

[Oskare_o]

The file crash info is empty or i don't know how to export it. I do the following:

admin@PA-2020> show system files

/var/cores/:
total 4.0K
drwxrwxrwx 2 root root 4.0K Oct 30 11:48 crashinfo

/var/cores/crashinfo:
total 16K
-rw-r--r-- 1 root root 16K Oct 30 11:48 kernel_panic_0

/opt/dpfs/var/cores/:
total 4.0K
drwxrwxrwx 2 root root 4.0K Mar  7  2014 crashinfo

/opt/dpfs/var/cores/crashinfo:
total 0
admin@PA-2020> tftp export core-file management-plane from crashinfo to 172.20.103.6
mode set to octet
Connected to 172.20.103.6 (172.20.103.6), port 69
putting /var/cores/crashinfo to 172.20.103.6:crashinfo [octet]

admin@PA-2020> tftp export core-file data-plane from crashinfo to 172.20.103.6

mode set to octet

Connected to 172.20.103.6 (172.20.103.6), port 69

putting /opt/dpfs/var/cores/crashinfo to 172.20.103.6:crashinfo [octet]

But the files ares empty, 0Kb.

This is the show system info output:

hostname: PA-2020
ip-address: 192.168.48.183
netmask: 255.255.255.0
default-gateway: 192.168.48.254
ipv6-address:
ipv6-link-local-address: fe80::21b:17ff:fe79:b700/64
ipv6-default-gateway:
mac-address: 00:1b:17:79:b7:00
time: Thu Oct 30 13:53:16 2014
uptime: 0 days, 1:48:54
family: 2000
model: PA-2020
serial: 0004C104280
sw-version: 5.0.9
global-protect-client-package-version: 0.0.0
app-version: 466-2435
app-release-date: 2014/10/28  20:28:09
av-version: 1401-1873
av-release-date: 2014/10/24  04:00:01
threat-version: 466-2435
threat-release-date: 2014/10/28  20:28:09
wildfire-version: 0
wildfire-release-date: unknown
url-filtering-version: 4111
global-protect-datafile-version: 0
global-protect-datafile-release-date: unknown
logdb-version: 5.0.2
platform-family: 2000
logger_mode: False
vpn-disable-mode: off
operational-mode: normal
multi-vsys: off

HULK

Before that logs there is this:

2014/10/30 08:03:52 info     userid         connect 0  ldap cfg Group01 connected to server 192.168.48.212:3268, initiated by: 192.1

68.48.183

2014/10/30 09:03:54 info     userid         connect 0  ldap cfg Group01 connected to server 192.168.48.212:3268, initiated by: 192.1

68.48.183

2014/10/30 10:03:56 info     userid         connect 0  ldap cfg Group01 connected to server 192.168.48.212:3268, initiated by: 192.1

68.48.183

2014/10/30 10:43:30 info     general        auth-su 0  User 'oortiz' authenticated.   From: 172.20.103.6.

2014/10/30 10:43:31 info     general        general 0  User oortiz logged in via Web from 172.20.103.6 using https

2014/10/30 10:43:31 info     general        general 0  Session for user lolivares via Web from 172.20.103.3 timed out

2014/10/30 11:03:58 info     userid         connect 0  ldap cfg Group01 connected to server 192.168.48.212:3268, initiated by: 192.1

68.48.183

2014/10/30 11:04:56 info     general        general 0  Connection to Update server: updates.paloaltonetworks.com completed successfu

lly, initiated by 192.168.48.183

2014/10/30 11:59:24 info     general        general 0  Management server started. Running version 5.0.9

2014/10/30 11:59:24 info     general        general 0  VPN Disable mode = off

2014/10/30 11:59:26 info     ntpd           restart 0  NTP restart synchronization performed

Thank you alot!

[HULK]

Thanks for your update. It looks like, there is no relevant SYSTEM logs during the incident. It would be better to contact support and let them analyze the tech-support file for root cause.

Thanks

[hshah]

Hi Oskare,

Firewall generated kernal panic at the time of reboot, which means its hitting one or other bug.

Please provide me output for following command. Those commands are used to read crash info files.

1. less mp-backtrace kernel_panic_0

2. less mp-backtrace kernel_panic_0  >>> Most likely you will get error with one of this two commands.

Regards,

Hardik Shah

[Oskare_o]

hshah

I shared to you a file with less mp-backtrace kernel_panic_0

Thanks you!!!

[hshah]

Hi OSkare,

Thanks for following file, I will update you soon

https://live.paloaltonetworks.com/docs/DOC-8259)

Regards,

Hardik Shah

[hshah · Accepted solution]

Hi Oskare,

I see following error in crash info.

Applying R_ERR on DMA activate FIS errata fix

This may or may not be a HDD RMA, I would suggest you to contact TAC and verify the same.

Regards,

Hardik Shah

[HULK]

Hello Oskare_o,

From the back trace it looks like the firewall was unable to read filesystem from the HDD drive during that time. It is showing the  ATA link error and to recover from this situation it got rebooted automatically.

3>ata1.00: failed command: READ DMA >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ATA link error

<3>ata1.00: cmd c8/00:08:f1:44:25/00:00:00:00:00/ec tag 0 dma 4096 in

<3>         res 40/00:00:01:4f:c2/00:00:00:00:00/00 Emask 0x56 (ATA bus error)

<3>ata1.00: status: { DRDY }

<6>ata1: hard resetting link

<3>ata1: failed to resume link (SControl FFFFFFFF)

<6>ata1: SATA link down (SStatus FFFFFFFF SControl FFFFFFFF)  >>>>>>>>>>>>>>>>>>>>>> link to the HDD was down

<6>ata1: hard resetting link

<3>ata1: failed to resume link (SControl FFFFFFFF)

<6>ata1: SATA link down (SStatus FFFFFFFF SControl FFFFFFFF)

<6>ata1: hard resetting link

<3>ata1: failed to resume link (SControl FFFFFFFF)

<6>ata1: SATA link down (SStatus FFFFFFFF SControl FFFFFFFF)

<4>ata1.00: disabled

<6>sd 0:0:0:0: [sda] Result: hostbyte=0x00 driverbyte=0x08

<6>sd 0:0:0:0: [sda] Sense Key : 0xb [current] [descriptor]

<4>Descriptor sense data with sense descriptors (in hex):

<6>        72 0b 00 00 00 00 00 0c 00 0a 80 00 00 00 00 00

<6>        00 00 00 00

<6>sd 0:0:0:0: [sda] ASC=0x0 ASCQ=0x0

<6>sd 0:0:0:0: [sda] CDB: cdb[0]=0x28: 28 00 0c 25 44 f1 00 00 08 00

<3>end_request: I/O error, dev sda, sector 203769073

<6>ata1: EH complete

<6>ata1.00: detaching (SCSI 0:0:0:0)

<1>Read-error on swap-device (8:0:44701239)

<1>Read-error on swap-device (8:0:44841087)

<1>Read-error on swap-device (8:0:44841103)

<1>Read-error on swap-device (8:0:44841111)

<1>Read-error on swap-device (8:0:44841119)

<1>Read-error on swap-device (8:0:44241407)

<1>Read-error on swap-device (8:0:44241415)

<1>Read-error on swap-device (8:0:44241423)

<1>Read-error on swap-device (8:0:44241431)

<1>Read-error on swap-device (8:0:44241447)

<1>Read-error on swap-device (8:0:44241455)

<1>Read-error on swap-device (8:0:44241463)

<1>Read-error on swap-device (8:0:44193367)

<1>Read-error on swap-device (8:0:44193375)

<1>Read-error on swap-device (8:0:44193383)

<1>Read-error on swap-device (8:0:44193399)

I would suggest you to contact PAN support and you may need to replace the HDD on this FW..

Hope this helps.

Thanks

[Oskare_o]

Thank for your help, I'm going to contact my suport partner,

Thank you very much!!

Saludos