PA is sluggish

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Announcements

PA is sluggish

L4 Transporter

I have removed a lot of rules and am down to 400 rules, I am up to OS 7.0.10 and upgrade the OS every other month

In the las couple months the PA seems to be getting slow and not as quit to commit changes or to pull up the dashboard when I first login. Any ideas where to look for possbile issues?

10 REPLIES 10

L6 Presenter

What's your platform?

 

Any increase in service usage, such as increased SSL decryption, logging, or user attribution being performed on box?  How many current sessions on average is the box handling?

Not sure what you mean by platform, its a PA 5050 OS 7.0.10. Session are very low right now due to it being a break( 39831/2000000).  Not doing any decryption. No user attribution 

 

 

you could try checking the system resources for 'abnormalities' or high load

 

do 'show system resources follow', then hit '1' to display all MP cpu's

the important ones are 'id' (idle) and 'wa' (wait)

if wait is high, there's a buffering issue which could indicate harddrive issues (either 'broken' or high IO due to log writing for example)

if one of the other stages is at 100%, something is using up all your cpu cycles

 

next up: mem and swap need at least a little 'free', if free = 0, there's more memory being used than there is available

 

lastly the individual daemons may help you pinpoint one that is taking up too many resources and could be 'helped along' by restarting it

 

admin@myNGFW> show system resources follow 


top - 09:52:02 up 1 day, 17:34,  1 user,  load average: 0.05, 0.07, 0.08
Tasks: 110 total,   1 running, 109 sleeping,   0 stopped,   0 zombie
Cpu0  :  0.7%us,  0.7%sy,  0.0%ni, 98.0%id,  0.0%wa,  0.0%hi,  0.7%si,  0.0%st
Cpu1  :  0.7%us,  0.0%sy,  0.0%ni, 99.3%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   3849936k total,  3095016k used,   754920k free,   248732k buffers
Swap:  3056660k total,   191424k used,  2865236k free,   609660k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                     
 2087 root      20   0 1131m 5416 2396 S  0.3  0.1   0:23.66 sysdagent                                                                                                    
 2966 root      20   0  808m 113m  10m S  0.3  3.0   0:53.32 devsrvr                                                                                                      
 3038 root      20   0  921m 301m 8276 S  0.3  8.0   6:19.41 mgmtsrvr                                                                                                     
    1 root      20   0 16548  632  600 S  0.0  0.0   0:01.49 init                                                                                                         
    2 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kthreadd                                                                                                     
    3 root      20   0     0    0    0 S  0.0  0.0   0:10.99 ksoftirqd/0                                                                                                  
    5 root       0 -20     0    0    0 S  0.0  0.0   0:00.00 kworker/0:0H                                                                                                 
    7 root      RT   0     0    0    0 S  0.0  0.0   0:00.87 migration/0                                                                                                  
    8 root      20   0     0    0    0 S  0.0  0.0   0:00.00 rcu_bh                                                                                                       
    9 root      20   0     0    0    0 S  0.0  0.0   0:09.73 rcu_sched                                                                                                    
   10 root      RT   0     0    0    0 S  0.0  0.0   0:00.91 migration/1                                                                                                  
   11 root      20   0     0    0    0 S  0.0  0.0   0:06.07 ksoftirqd/1                                                                                                  
   12 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kworker/1:0                         
Tom Piens
PANgurus - Strata specialist; config reviews, policy optimization

I will check it out and post what I find

show system resources follow

 

 3552 root      20   0 1226m 141m 5292 S  4.0  3.6   3586:27 logrcvr

18614 root      20   0  2164  716  592 S  2.0  0.0   0:00.06 ping

 2410 root      15  -5 24820 7004 1712 S  0.3  0.2 213:02.00 sysd

17286 nobody    20   0  113m 7396 4316 S  0.3  0.2  79:00.06 appweb3

18440 jprovine  20   0  2540 1136  836 R  0.3  0.0   0:00.12 top

    1 root      20   0  2084  556  536 S  0.0  0.0   0:17.88 init

    2 root      20   0     0    0    0 S  0.0  0.0   0:00.02 kthreadd

    3 root      RT   0     0    0    0 S  0.0  0.0   0:10.28 migration/0

    4 root      20   0     0    0    0 S  0.0  0.0   1:21.10 ksoftirqd/0

    5 root      RT   0     0    0    0 S  0.0  0.0   0:10.22 migration/1

    6 root      20   0     0    0    0 S  0.0  0.0   0:26.35 ksoftirqd/1

    7 root      RT   0     0    0    0 S  0.0  0.0   0:11.90 migration/2

    8 root      20   0     0    0    0 S  0.0  0.0   0:25.33 ksoftirqd/2

    9 root      RT   0     0    0    0 S  0.0  0.0   0:10.82 migration/3

   10 root      20   0     0    0    0 S  0.0  0.0   0:27.66 ksoftirqd/3

   11 root      20   0     0    0    0 S  0.0  0.0   1:08.05 events/0

   12 root      20   0     0    0    0 S  0.0  0.0   7:28.57 events/1

   13 root      20   0     0    0    0 S  0.0  0.0   4:05.17 events/2

   14 root      20   0     0    0    0 S  0.0  0.0   9:52.74 events/3

   15 root      20   0     0    0    0 S  0.0  0.0   0:00.01 khelper

   20 root      20   0     0    0    0 S  0.0  0.0   0:00.00 async/mgr

  166 root      20   0     0    0    0 S  0.0  0.0   0:02.27 sync_supers

  168 root      20   0     0    0    0 S  0.0  0.0   0:02.82 bdi-default

  169 root      20   0     0    0    0 S  0.0  0.0   0:27.91 kblockd/0

  170 root      20   0     0    0    0 S  0.0  0.0   0:20.63 kblockd/1

  171 root      20   0     0    0    0 S  0.0  0.0   0:18.41 kblockd/2

  172 root      20   0     0    0    0 S  0.0  0.0   0:24.93 kblockd/3

  174 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kacpid

  175 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kacpi_notify

  176 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kacpi_hotplug

  304 root      20   0     0    0    0 S  0.0  0.0   0:00.00 ata/0

  305 root      20   0     0    0    0 S  0.0  0.0   0:00.00 ata/1

  306 root      20   0     0    0    0 S  0.0  0.0   0:00.00 ata/2

  307 root      20   0     0    0    0 S  0.0  0.0   0:00.00 ata/3

  308 root      20   0     0    0    0 S  0.0  0.0   0:00.00 ata_aux

  309 root      20   0     0    0    0 S  0.0  0.0   0:00.00 ksuspend_usbd

  314 root      20   0     0    0    0 S  0.0  0.0   0:00.00 khubd

  317 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kseriod

  351 root      20   0     0    0    0 S  0.0  0.0   0:00.00 rpciod/0

  352 root      20   0     0    0    0 S  0.0  0.0   0:00.00 rpciod/1

  353 root      20   0     0    0    0 S  0.0  0.0   0:00.00 rpciod/2

  354 root      20   0     0    0    0 S  0.0  0.0   0:00.00 rpciod/3

  388 root      20   0     0    0    0 S  0.0  0.0  34:10.42 kswapd0

  389 root      20   0     0    0    0 S  0.0  0.0   0:00.00 aio/0

  390 root      20   0     0    0    0 S  0.0  0.0   0:00.00 aio/1

  391 root      20   0     0    0    0 S  0.0  0.0   0:00.00 aio/2

  392 root      20   0     0    0    0 S  0.0  0.0   0:00.00 aio/3

  393 root       1 -19     0    0    0 S  0.0  0.0   0:00.00 nfsiod

  429 root      20   0     0    0    0 S  0.0  0.0   0:00.00 pciehpd

  568 root      20   0     0    0    0 S  0.0  0.0   0:00.00 scsi_eh_0

your logrcvr is running pretty high on VIRT memory, might try restarting that process to see if it helps at all

 

> debug software restart process log-receiver 
Tom Piens
PANgurus - Strata specialist; config reviews, policy optimization

So it is the logging that is causes the slowness on the firewall and that would be related to the management plane wouldn't it

well the logging process is taking up a lot of your memory, so i would call it suspect, not necessarily 'the culprit'

it lives on the management plane so that's certainly a good indication 🙂

 

if you restart the service (grab a techsupport file beforehand, maybe also 'core' the process during the restart) and your GUI experience improves, that's that. Else we need to keep investigating 🙂

Tom Piens
PANgurus - Strata specialist; config reviews, policy optimization

So restarting the process during regular operation won't cause any issues or interfer with its operation?

not in your normal operations

log entries may get backlogged or lost (depending on the volume of log you generate, the dataplane buffer may nog be enough)

but sessions will not be interrupted

Tom Piens
PANgurus - Strata specialist; config reviews, policy optimization
  • 3175 Views
  • 10 replies
  • 0 Likes
Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!

The LIVEcommunity thanks you for your participation!