Zone Protection drops traffic
cancel
Showing results for 
Search instead for 
Did you mean: 

Zone Protection drops traffic

L4 Transporter

We have below settings for  our untrust zone protection. We don't see a high CPS rate but we still see packets getting dropped, and has now started effecting us. Any guidance would be helpful.

PANO9.0.11/5250

 

image.png

image.png

image.png

 

 

9 REPLIES 9

L7 Applicator

during a time you're seeing these drops, see if you are picking up syn cookie error counters:

 

> show counter global delta yes filter aspect dos

 

and look for any of these counters:

flow_dos_syncookie_ack_err info TCP SYN cookies: Invalid ACKs received, aggregate profile/zone
flow_dos_syncookie_blk_dur drop Packets dropped: Flagged for blocking and under block duration for ag
flow_dos_syncookie_max drop Packet dropped: SYN cookies maximum threshold reached, aggregate prof
flow_dos_syncookie_not_tcp_syn drop TCP SYN cookies: TCP SYN cookie not SYN
flow_dos_syncookie_not_tcp_syn_ack drop TCP SYN cookies: TCP SYN cookie not SYN-ACK

Tom Piens
Like my answer? check out my book! https://bit.ly/MasteringPAN

@reaper  Its difficult to so when it happens as it lasts only few minutes, and not predictable when it would happen.

But because it had been effecting users I had to increase the activation to rate to 4000.

Below is the output though and i only see 1 flow that matches what you said to look for. flow_dos_syncookie_ack_er

 

show counter global filter delta yes aspect dos

Global counters:
Elapsed time since last sampling: 5.684 seconds

name                                   value     rate severity  category  aspect    description
--------------------------------------------------------------------------------
flow_dos_syncookie_cookie_sent          2170      381 info      flow      dos       TCP SYN cookies: cookies sent, aggregate profile/zone
flow_dos_syncookie_ack_rcv              2332      410 info      flow      dos       TCP SYN cookies: ACKs to cookies received, aggregate profile/zone
flow_dos_syncookie_ack_err                81       14 info      flow      dos       TCP SYN cookies: Invalid ACKs received, aggregate profile/zone
flow_dos_syncookie_svr_ack_rcv          2076      365 info      flow      dos       TCP SYN cookies: Server ACKs received, aggregate profile/zone
flow_dos_syncookie_not_tcp_syn           304       53 drop      flow      dos       TCP SYN cookies: TCP SYN cookie not SYN
flow_dos_pf_ipspoof                        1        0 drop      flow      dos       Packets dropped: Zone protection option 'discard-ip-spoof'
flow_dos_pf_ipfrag                        24        4 drop      flow      dos       Packets dropped: Zone protection option 'discard-ip-frag'
flow_dos_pf_ping0                         10        1 drop      flow      dos       Packets dropped: Zone protection option 'discard-icmp-ping-zero-id'
flow_dos_pf_icmplpkt                       1        0 drop      flow      dos       Packets dropped: Zone protection option 'discard-icmp-large-packet'
flow_dos_pm_tcptimestamp                  65       11 info      flow      dos       Packets modified: Zone protection option 'remove-tcp-timestamp'
flow_dos_rule_allow_under_rate            24        4 info      flow      dos       Packets allowed: Rate within thresholds of DoS policy
flow_dos_rule_match                       24        4 info      flow      dos       Packets matched DoS policy
flow_dos_rule_nomatch                   3800      668 info      flow      dos       Packets not matched DoS policy
flow_dos_cl_curr_sess_add_incr            14        2 info      flow      dos       Incremented classified current session count on session create
flow_dos_cl_curr_sess_del_decr            30        5 info      flow      dos       Decremented classified current session count on session delete
flow_dos_ag_buckets_upd                   11        1 info      flow      dos       Updated aggregate buckets for aging
--------------------------------------------------------------------------------
Total counters shown: 16
--------------------------------------------------------------------------------

L7 Applicator

Check to see if you have a lot of non_syn_tcp

 

 

show counter global filter aspect session category flow severity drop delta yes

Tom Piens
Like my answer? check out my book! https://bit.ly/MasteringPAN

@raji_toor,

If you are having trouble checking counters when experiencing the issue, I would simply script it and log the results on a regular basis. That way, when you experience the issue again you simply have to go back and look at the logged data to see if you can capture the counters increasing at a high rate. 

@BPry  Would you have a sample script for just that.

 

@reaper This seems quite bursty 

show counter global filter aspect session category flow severity drop delta yes

Global counters:
Elapsed time since last sampling: 8.763 seconds

name                                   value     rate severity  category  aspect    description
--------------------------------------------------------------------------------
flow_policy_deny                        3575      407 drop      flow      session   Session setup: denied by policy
flow_tcp_non_syn_drop                  19507     2226 drop      flow      session   Packets dropped: non-SYN TCP without session match
--------------------------------------------------------------------------------
Total counters shown: 2
--------------------------------------------------------------------------------




Global counters:
Elapsed time since last sampling: 1.548 seconds

name                                   value     rate severity  category  aspect    description
--------------------------------------------------------------------------------
flow_policy_deny                         636      410 drop      flow      session   Session setup: denied by policy
flow_tcp_non_syn_drop                   3530     2280 drop      flow      session   Packets dropped: non-SYN TCP without session match
--------------------------------------------------------------------------------
Total counters shown: 2
--------------------------------------------------------------------------------

 

 

Also from what i have understood these are global counters, and they are just telling something is happening but don't tell where. Could the counters be not increasing because of something internal to our network for these counters. 

 

This is what I seen with show zone-protection 

 

 show zone-protection zone EXTERNAL

------------------------------------------------------------------------------------------
Number of zones with protection profile: 1
------------------------------------------------------------------------------------------
Zone EXTERNAL, vsys vsys1, profile EXTERNAL
------------------------------------------------------------------------------------------
  tcp-syn              SYN cookie enabled: yes
    DP alarm rate:        7000 cps, activate rate:    4500 cps, maximal rate:   35000 cps
    current:                84 packets
    dropped:                 0 packets
    not TCP SYN:       196153882 packets
    not TCP SYN ACK:     25849 packets
------------------------------------------------------------------------------------------
  udp                  RED enabled: yes
    DP alarm rate:        5000 cps, activate rate:   15000 cps, maximal rate:   30000 cps
    current:                 2 packets
    dropped:                 0 packets
------------------------------------------------------------------------------------------
  icmp                 RED enabled: yes
    DP alarm rate:        5000 cps, activate rate:   15000 cps, maximal rate:   30000 cps
    current:                 0 packets
    dropped:                 0 packets
------------------------------------------------------------------------------------------
  other-ip             RED enabled: yes
    DP alarm rate:        5000 cps, activate rate:   15000 cps, maximal rate:   30000 cps
    current:                 0 packets
    dropped:                 0 packets
------------------------------------------------------------------------------------------
  icmpv6               RED enabled: yes
    DP alarm rate:        5000 cps, activate rate:   15000 cps, maximal rate:   30000 cps
    current:                 0 packets
    dropped:                 0 packets
------------------------------------------------------------------------------------------
IPv(4/6) Filter:
  discard-ip-spoof:                           enabled: yes, packet dropped: 121
  discard-ip-frag:                            enabled: yes, packet dropped: 3146
  tcp-reject-non-syn:                         enabled: yes, (global), packet dropped: 108198
  tcp-timestamp:                              enabled: yes, packets modified: 4370
  discard-tcp-syn-with-data:                  enabled: yes, packet dropped: 0
  discard-tcp-synack-with-data:               enabled: yes, packet dropped: 0
  strip-tcp-fast-open-and-data:               enabled: yes, packet stripped: 9
IPv4 packet filter:
  discard-icmp-ping-zero-id:                  enabled: yes, packet dropped: 30
  discard-icmp-frag:                          enabled: yes, packet dropped: 0
  discard-icmp-large-packet:                  enabled: yes, packet dropped: 0

 

I have made this powershell script to get  zone information . There might be easy ways, but this is what I got to work and log the output

 

#Loop endlessly with 5 second delay w
while($true){
$Login = Invoke-restmethod -Uri 'https://pan/api/?type=keygen&user=xml&password=P@ssword1' 
$LoginXML = Select-Xml -Xml $Login -XPath "/response[@status]/result/key/text()" | Select-Object -ExpandProperty Node
$Key = $LoginXML.Value
$XMLURI =  'https://pan/api/?type=op&cmd=<show><zone-protection><zone>EXTERNAL</zone></zone-protection></show>&key=' + $Key
$XMLURI
$Request = Invoke-restmethod -Uri $XMLURI
$Parse0 = Select-Xml -Xml $Request -XPath "/response[@status]/result/entry/entries/entry" | Select-Object -ExpandProperty Node
$Parse1 = Select-Xml -Xml $Request -XPath "/response[@status]/result/entry/entries/entry/other-ip" | Select-Object -ExpandProperty Node
$Parse2 = Select-Xml -Xml $Request -XPath "/response[@status]/result/entry/entries/entry/tcp-syn" | Select-Object -ExpandProperty Node
$Parse3 = Select-Xml -Xml $Request -XPath "/response[@status]/result/entry/entries/entry/udp" | Select-Object -ExpandProperty Node
$Parse4 = Select-Xml -Xml $Request -XPath "/response[@status]/result/entry/entries/entry/icmp" | Select-Object -ExpandProperty Node
$Parse5 = Select-Xml -Xml $Request -XPath "/response[@status]/result/entry/entries/entry/icmpv6" | Select-Object -ExpandProperty Node

$Time = (Get-Date).ToString(‘yyyy-MM-dd HH:mm:ss’)

$Parse0  | Add-Member -MemberType NoteProperty -Name Timestamp -Value $Time -Passthru | Export-Csv -Path "d:\palo\Entry.csv" -Append
$Parse1 | Add-Member -MemberType NoteProperty -Name Timestamp -Value $Time -Passthru | Export-Csv -Path "d:\palo\OTHER-IP.csv" -Append
$Parse2 | Add-Member -MemberType NoteProperty -Name Timestamp -Value $Time -Passthru | Export-Csv -Path "d:\palo\TCP-SYN.csv" -Append
$Parse3 | Add-Member -MemberType NoteProperty -Name Timestamp -Value $Time -Passthru | Export-Csv -Path "d:\palo\UDP.CSV" -Append
$Parse4 | Add-Member -MemberType NoteProperty -Name Timestamp -Value $Time -Passthru | Export-Csv -Path "d:\palo\ICMP.csv" -Append
$Parse5 | Add-Member -MemberType NoteProperty -Name Timestamp -Value $Time -Passthru | Export-Csv -Path "d:\palo\ICMP6.csv" -Append

Start-Sleep -s 5

}

 

@BPry  and @reaper  do you think if it benefits to log global counters

No, the potential to cause a catastrophic load on your device is too high IMHO

 

If you're at the end of your rope, open a case with support so they can assist during a live session

Tom Piens
Like my answer? check out my book! https://bit.ly/MasteringPAN

@raji_toor,

I've never had a need to programmatically pull global counters on an automated basis outside of specific troubleshooting periods where I was already in maintenance windows. I'm not sure if it would put enough load on the firewall to cause any issues; I don't recall it placing an abnormal load on the system in the past, but I was also in maintenance windows and wouldn't really have been looking for it. 

Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!

The LIVEcommunity thanks you for your participation!