07-08-2014 08:03 AM
We have several PAN 3020s at a client site with similar issues but for this, I’ll focus on a specific case. One pair in Active\Passive HA has 124 rules. We started noticing really slow RDP connect performance. (it would take 45 seconds to establish an RDP session to a target where the traffic was passed through the firewall). Out of the 124 rules, the rule which this RDP traffic matched on was around rule 100. If we moved that rule up earlier in the ACL to say, rule 5, the RDP session would only take 10 seconds or less to establish.
So initially, it is looking like the further down the ACL the rule is, the longer it takes the PAN to process that traffic. However, seeing that the 3020 supports up to 2500 policies and we only have 124, I wanted to check with you as it doesn’t seem right.
We are not doing any PBF here. Or App-ID override. Jumbo frames are enabled.
07-08-2014 08:15 AM
Hello mackwage,
Would it be possible for you to take a packet capture for this traffic in both "receive" and "Transmit" stage. Then we can compare the processing time between receive and transmit in both conditions ( Rule at 5 and rule at 100).
What is the Data-plane utilization on this firewall including packet-rate, total active sessions, H/W nd S/W buffer, packet-descriptor, DP-CPU etc..?
Thanks
07-08-2014 08:23 AM
Thanks for the reply! We can work on getting pcaps.
For the dataplane utilization, I am not sure where to get ALL of that info.
In the WebGUI, I can see the DP-CPU ranges between 2-6%. We ran show system statistics session and got.
The average throughput was 400-800k.
The sessions were ~900.
07-08-2014 08:27 AM
Hi Mack,
Firewall is under utilized, hence its not over utilization issue. You are right, you can view MP/DP CPU on Dashboard.
Please provide us following output.
debug data-plane pool statistics
show system statistics session
show running resource-monitor
show session info
show counter global filter delta yes ( repeat the same command 5 times)
Regards,
Hardik shah
07-08-2014 08:32 AM
PCAP along with the resource utilization would give us more insight about this issue.
Thanks
07-08-2014 08:58 AM
debug data-plane pool statistics
Hardware Pools
[ 0] Packet Buffers : 11341/11468 0x8000000031000000
[ 1] Work Queue Entries : 229309/229376 0x8000000037ffe000
[ 2] Output Buffers : 1006/1024 0x8000000039bfe000
[ 3] DFA Result : 3999/4000 0x8000000039cfe000
[ 4] Timer Buffers : 4092/4096 0x800000003a0e6000
[ 5] PAN_FPA_LWM_POOL : 1024/1024 0x8000000000e9f200
[ 6] PAN_FPA_ZIP_POOL : 1023/1024 0x800000003a4e6000
[ 7] PAN_FPA_BLAST_POOL : 1024/1024 0x800000000ff00000
Software Pools
[ 0] software packet buffer 0 : 32767/32768 0x800000003a6e6680
[ 1] software packet buffer 1 : 32768/32768 0x800000003b706700
[ 2] software packet buffer 2 : 81920/81920 0x800000003d726780
[ 3] software packet buffer 3 : 20480/20480 0x8000000047776800
[ 4] software packet buffer 4 : 304/304 0x800000007018a880
[ 5] ZIP Results : 1024/1024 0x8000000084d4c0e0
[ 6] CTD Flow : 261635/262144 0x8000000084d66080
[ 7] CTD AV Block : 32/32 0x80000000a20bd340
[ 8] SML VM Fields : 524001/524288 0x80000000a20c5440
[ 9] SML VM Vchecks : 65536/65536 0x80000000a32c54c0
[10] Detector Threats : 196189/196608 0x80000000a3405540
[11] CTD DLP FLOW : 65536/65536 0x80000000a5da5608
[12] CTD DLP DATA : 4096/4096 0x80000000a65e5688
[13] CTD DECODE FILTER : 65536/65536 0x80000000a69e9710
[14] Regex Results : 8000/8000 0x80000000a6d4a088
[15] TIMER Chunk : 131072/131072 0x80000000aeca5ae0
[16] FPTCP segs : 32768/32768 0x80000000b0d25b60
[17] Proxy session : 7936/7936 0x80000000b0dc5be0
[18] SSL Handshake State : 7936/7936 0x80000000b1096860
[19] SSL State : 15872/15872 0x80000000b19564e0
[20] SSL Handshake MAC State : 17464/17464 0x80000000b25a0d60
[21] SSH Handshake State : 64/64 0x80000000b27d3ac0
[22] SSH State : 512/512 0x80000000b2841640
[23] TCP host connections : 15/16 0x80000000b297d020
show system statistics session
Device is up : 7 days 0 hour 39 mins 39 sec
Packet rate : 196/s
Throughput : 479 Kbps
Total active sessions : 818
Active TCP sessions : 503
Active UDP sessions : 311
Active ICMP sessions : 4
show running resource-monitor
Resource monitoring sampling data (per second):
CPU load sampling by group:
flow_lookup : 5%
flow_fastpath : 5%
flow_slowpath : 5%
flow_forwarding : 5%
flow_mgmt : 1%
flow_ctrl : 1%
nac_result : 5%
flow_np : 5%
dfa_result : 5%
module_internal : 5%
aho_result : 5%
zip_result : 5%
pktlog_forwarding : 6%
lwm : 0%
flow_host : 1%
show session info
--------------------------------------------------------------------------------
Number of sessions supported: 262142
Number of active sessions: 845
Number of active TCP sessions: 505
Number of active UDP sessions: 330
Number of active ICMP sessions: 10
Number of active BCAST sessions: 0
Number of active MCAST sessions: 0
Number of active predict sessions: 0
Session table utilization: 0%
Number of sessions created since bootup: 4899529
Packet rate: 296/s
Throughput: 955 kbps
New connection establish rate: 0 cps
--------------------------------------------------------------------------------
Session timeout
TCP default timeout: 3600 secs
TCP session timeout before SYN-ACK received: 5 secs
TCP session timeout before 3-way handshaking: 10 secs
TCP session timeout after FIN/RST: 30 secs
UDP default timeout: 30 secs
ICMP default timeout: 6 secs
other IP default timeout: 30 secs
Captive Portal session timeout: 30 secs
Session timeout in discard state:
TCP: 90 secs, UDP: 60 secs, other IP protocols: 60 secs
--------------------------------------------------------------------------------
Session accelerated aging: True
Accelerated aging threshold: 80% of utilization
Scaling factor: 2 X
--------------------------------------------------------------------------------
Session setup
TCP - reject non-SYN first packet: True
Hardware session offloading: True
IPv6 firewalling: True
--------------------------------------------------------------------------------
Application trickling scan parameters:
Timeout to determine application trickling: 10 secs
Resource utilization threshold to start scan: 80%
Scan scaling factor over regular aging: 8
--------------------------------------------------------------------------------
Session behavior when resource limit is reached: drop
--------------------------------------------------------------------------------
Pcap token bucket rate : 10485760
--------------------------------------------------------------------------------
07-08-2014 08:59 AM
I would take a look at the rules and organization. One of the things I've seen that can affect initial session setup time is the number of rules that are application only with the "any" port selection.
These rules should always be towards the bottom of the rule base.
The most specific rules with specified ports and applications should be at the top.
What happens with application and any rules is that PanOS essentially holds on that rule while the traffic is coming through and makes sure that the application is not a match before moving on to the next rule for checks. If there are a number of these "any" port rules each one does the check and passes on and the time delay can add up.
07-08-2014 09:00 AM
show counter global filter delta yes ( repeat the same command 5 times)
Global counters:
Elapsed time since last sampling: 156.760 seconds
name value rate severity category aspect description
--------------------------------------------------------------------------------
pkt_recv 569924 3635 info packet pktproc Packets received
pkt_recv_zero 569190 3630 info packet pktproc Packets received from QoS 0
pkt_sent 559573 3569 info packet pktproc Packets transmitted
pkt_alloc 588942 3756 info packet resource Packets allocated
session_allocated 846 5 info session resource Sessions allocated
session_freed 874 5 info session resource Sessions freed
session_installed 843 5 info session resource Sessions installed
session_discard 3 0 info session resource Session set to discard by security policy check
flow_rcv_dot1q_tag_err 247 1 drop flow parse Packets dropped: 802.1q tag not configured
flow_no_interface 247 1 drop flow parse Packets dropped: invalid interface
flow_ipv6_disabled 471 3 drop flow parse Packets dropped: IPv6 disabled on interface
flow_policy_deny 176 1 drop flow session Session setup: denied by policy
flow_tcp_non_syn 52 0 info flow session Non-SYN TCP packets without session match
flow_tcp_non_syn_drop 52 0 drop flow session Packets dropped: non-SYN TCP without session match
flow_fwd_l3_bcast_drop 18 0 drop flow forward Packets dropped: unhandled IP broadcast
flow_fwd_l3_mcast_drop 376 2 drop flow forward Packets dropped: no route for IP multicast
flow_fwd_l3_noarp 58 0 drop flow forward Packets dropped: no ARP
flow_fwd_zonechange 31 0 drop flow forward Packets dropped: forwarded to different zone
flow_parse_unmatched_icmperr 77 0 info flow parse Packets dropped: Unmatched ICMP error message
flow_dos_rule_allow_under_rate 6 0 info flow dos Packets allowed: Rate within thresholds of DoS policy
flow_dos_rule_match 6 0 info flow dos Packets matched DoS policy
flow_dos_rule_nomatch 840 5 info flow dos Packets not matched DoS policy
flow_dos_ag_curr_sess_add_incr 6 0 info flow dos Incremented aggregate current session count on session create
flow_dos_ag_curr_sess_del_decr 5 0 info flow dos Decremented aggregate current session count on session delete
flow_dos_cl_curr_sess_add_incr 6 0 info flow dos Incremented classified current session count on session create
flow_dos_cl_curr_sess_del_decr 5 0 info flow dos Decremented classified current session count on session delete
flow_dos_ag_buckets_upd 314 2 info flow dos Updated aggregate buckets for aging
flow_action_close 2 0 drop flow pktproc TCP sessions closed via injecting RST
flow_arp_pkt_rcv 178 1 info flow arp ARP packets received
flow_arp_pkt_xmt 122 0 info flow arp ARP packets transmitted
flow_arp_pkt_replied 114 0 info flow arp ARP requests replied
flow_arp_rcv_gratuitous 19 0 info flow arp Gratuitous ARP packets received
flow_arp_resolve_xmt 26 0 info flow arp ARP resolution packets transmitted
flow_host_pkt_rcv 734 4 info flow mgmt Packets received from control plane
flow_host_pkt_xmt 1437 9 info flow mgmt Packets transmitted to control plane
flow_host_service_allow 95 0 info flow mgmt Device management session allowed
flow_health_monitor_rcv 673 4 info flow mgmt Health monitoring packet received
flow_health_monitor_xmt 673 4 info flow mgmt Health monitoring packet transmitted
appid_ident_by_icmp 201 1 info appid pktproc Application identified by icmp type
appid_ident_by_dport_first 156 0 info appid pktproc Application identified by L4 dport first
appid_ident_by_dport 12 0 info appid pktproc Application identified by L4 dport
appid_proc 443 2 info appid pktproc The number of packets processed by Application identification
appid_use_dfa_1 258 1 info appid pktproc The number of packets using the second DFA table
appid_unknown_fini_empty 109 0 info appid pktproc The number of unknown applications because of no data
nat_static_xlat 4 0 info nat resource The total number of static NAT translate called
nat_static_release 6 0 info nat resource The total number of static NAT release called
nat_dynamic_port_xlat 15 0 info nat resource The total number of dynamic_ip_port NAT translate called
nat_dynamic_port_release 15 0 info nat resource The total number of dynamic_ip_port NAT release called
dfa_dte_request_total 147094 938 info dfa offload The total number of dfa match using dte
dfa_hte_in_cache_lookup 146276 933 info dfa offload The total number of requests to an in cache HFA graph
tcp_case_2 209 1 info tcp pktproc tcp reassembly case 2
ctd_sml_exit_detector_i 27 0 info ctd pktproc The number of sessions with sml exit in detector i
appid_bypass_no_ctd 12 0 info appid pktproc appid bypass due to no ctd
ctd_err_bypass 27 0 info ctd pktproc ctd error bypass
ctd_run_pattern_match_failure 20 0 info ctd pktproc Run pattern match failure
ctd_do_pattern_match 20 0 info ctd pktproc do pattern match
ctd_sml_vm_run_impl_opcodeexit 27 0 info ctd pktproc SML VM opcode exit
[64;1H [K [7mlines 1-63 [27m [64;1H [64;1H [Kctd_sml_vm_run_impl_immed8000 662 4 info ctd pktproc SML VM immed8000
ctd_sml_vm_check_domain 159 1 info ctd pktproc sml vm check domain
ctd_sml_opcode_set_file_type 688 4 info ctd pktproc sml opcode set file type
ctd_filter_decode_failure_zip 34 0 error ctd pktproc Number of decode filter failure for zip
ctd_bloom_filter_nohit 162 1 info ctd pktproc The number of no match for virus bloom filter
ctd_bloom_filter_hit 9 0 info ctd pktproc The number of match for virus bloom filter
ctd_bloom_filter_pattern_notfound 9 0 info ctd pktproc The number of missing pattern match for virus bloom filter
aho_fpga 297239 1896 info aho resource The total requests to FPGA for AHO
aho_sw 97 0 info aho pktproc The total usage of software for AHO
ctd_appid_reassign 48318 308 info ctd pktproc appid was changed
ctd_url_block 1 0 info ctd pktproc sessions blocked by url filtering
ctd_pkt_slowpath 149601 954 info ctd pktproc Packets processed by slowpath
ha_msg_sent 290908 1855 info ha system HA: messages sent
ha_msg_recv 313 1 info ha system HA: messages received
ha_session_setup_msg_sent 712 4 info ha pktproc HA: session setup messages sent
ha_session_teardown_msg_sent 398 2 info ha pktproc HA: session teardown messages sent
ha_session_update_msg_sent 289539 1847 info ha pktproc HA: session update messages sent
ha_arp_update_msg_sent 102 0 info ha pktproc HA: ARP update messages sent
ha_ha2_monitor_msg_sent 313 1 info ha pktproc HA: HA2 monitor message messages sent
ha_ha2_monitor_msg_recv 313 1 info ha pktproc HA: HA2 monitor message messages received
log_vulnerability_cnt 2066 13 info log system Number of vulnerability logs
log_fileext_cnt 18 0 info log system Number of file block logs
log_traffic_cnt 980 6 info log system Number of traffic logs
zip_process_total 104 0 info zip pktproc The total number of zip engine decompress process
zip_process_failure 34 0 info zip pktproc The number of failures for zip decompress process
pkt_nac_result 297239 1896 info packet resource Packets entered module nac stage result
pkt_flow_np 271951 1734 info packet resource Packets entered module flow stage np
pkt_flow_host 734 4 info packet resource Packets entered module flow stage host
--------------------------------------------------------------------------------
Total counters shown: 85
--------------------------------------------------------------------------------
Global counters:
Elapsed time since last sampling: 4.680 seconds
name value rate severity category aspect description
--------------------------------------------------------------------------------
pkt_recv 19543 4175 info packet pktproc Packets received
pkt_recv_zero 19522 4171 info packet pktproc Packets received from QoS 0
pkt_sent 17488 3736 info packet pktproc Packets transmitted
pkt_alloc 20124 4300 info packet resource Packets allocated
session_allocated 50 10 info session resource Sessions allocated
session_freed 18 3 info session resource Sessions freed
session_installed 50 10 info session resource Sessions installed
flow_rcv_dot1q_tag_err 6 1 drop flow parse Packets dropped: 802.1q tag not configured
flow_no_interface 6 1 drop flow parse Packets dropped: invalid interface
flow_ipv6_disabled 19 4 drop flow parse Packets dropped: IPv6 disabled on interface
flow_policy_deny 5 1 drop flow session Session setup: denied by policy
flow_fwd_l3_bcast_drop 2 0 drop flow forward Packets dropped: unhandled IP broadcast
flow_fwd_l3_mcast_drop 12 2 drop flow forward Packets dropped: no route for IP multicast
flow_fwd_l3_noarp 3 0 drop flow forward Packets dropped: no ARP
flow_fwd_zonechange 1 0 drop flow forward Packets dropped: forwarded to different zone
flow_parse_unmatched_icmperr 6 1 info flow parse Packets dropped: Unmatched ICMP error message
flow_dos_rule_nomatch 50 10 info flow dos Packets not matched DoS policy
flow_dos_ag_buckets_upd 9 1 info flow dos Updated aggregate buckets for aging
flow_arp_pkt_rcv 5 1 info flow arp ARP packets received
flow_arp_pkt_replied 1 0 info flow arp ARP requests replied
flow_host_pkt_rcv 21 4 info flow mgmt Packets received from control plane
flow_host_pkt_xmt 39 8 info flow mgmt Packets transmitted to control plane
flow_host_service_allow 4 0 info flow mgmt Device management session allowed
flow_health_monitor_rcv 18 3 info flow mgmt Health monitoring packet received
flow_health_monitor_xmt 18 3 info flow mgmt Health monitoring packet transmitted
appid_ident_by_icmp 12 2 info appid pktproc Application identified by icmp type
appid_ident_by_dport_first 1 0 info appid pktproc Application identified by L4 dport first
appid_proc 30 6 info appid pktproc The number of packets processed by Application identification
appid_use_dfa_1 13 2 info appid pktproc The number of packets using the second DFA table
appid_unknown_fini_empty 7 1 info appid pktproc The number of unknown applications because of no data
nat_static_xlat 2 0 info nat resource The total number of static NAT translate called
dfa_dte_request_total 5534 1182 info dfa offload The total number of dfa match using dte
dfa_hte_in_cache_lookup 5465 1167 info dfa offload The total number of requests to an in cache HFA graph
tcp_case_2 8 1 info tcp pktproc tcp reassembly case 2
ctd_sml_exit_detector_i 3 0 info ctd pktproc The number of sessions with sml exit in detector i
appid_bypass_no_ctd 3 0 info appid pktproc appid bypass due to no ctd
ctd_err_bypass 3 0 info ctd pktproc ctd error bypass
ctd_run_pattern_match_failure 1 0 info ctd pktproc Run pattern match failure
ctd_do_pattern_match 1 0 info ctd pktproc do pattern match
ctd_sml_vm_run_impl_opcodeexit 3 0 info ctd pktproc SML VM opcode exit
ctd_sml_vm_run_impl_immed8000 20 4 info ctd pktproc SML VM immed8000
ctd_sml_vm_check_domain 1 0 info ctd pktproc sml vm check domain
ctd_sml_opcode_set_file_type 21 4 info ctd pktproc sml opcode set file type
ctd_bloom_filter_nohit 1 0 info ctd pktproc The number of no match for virus bloom filter
aho_fpga 11017 2354 info aho resource The total requests to FPGA for AHO
aho_sw 14 2 info aho pktproc The total usage of software for AHO
ctd_appid_reassign 1871 399 info ctd pktproc appid was changed
ctd_pkt_slowpath 5573 1190 info ctd pktproc Packets processed by slowpath
ha_msg_sent 9091 1942 info ha system HA: messages sent
ha_msg_recv 10 2 info ha system HA: messages received
ha_session_setup_msg_sent 40 8 info ha pktproc HA: session setup messages sent
ha_session_teardown_msg_sent 8 1 info ha pktproc HA: session teardown messages sent
ha_session_update_msg_sent 9037 1930 info ha pktproc HA: session update messages sent
ha_arp_update_msg_sent 1 0 info ha pktproc HA: ARP update messages sent
ha_ha2_monitor_msg_sent 10 2 info ha pktproc HA: HA2 monitor message messages sent
ha_ha2_monitor_msg_recv 10 2 info ha pktproc HA: HA2 monitor message messages received
log_vulnerability_cnt 48 10 info log system Number of vulnerability logs
[64;1H [K [7mlines 1-63 [27m [64;1H [64;1H [Klog_fileext_cnt 4 0 info log system Number of file block logs
log_traffic_cnt 23 4 info log system Number of traffic logs
pkt_nac_result 11017 2354 info packet resource Packets entered module nac stage result
pkt_flow_np 8506 1817 info packet resource Packets entered module flow stage np
pkt_flow_host 21 4 info packet resource Packets entered module flow stage host
--------------------------------------------------------------------------------
Total counters shown: 62
--------------------------------------------------------------------------------
Global counters:
Elapsed time since last sampling: 3.820 seconds
name value rate severity category aspect description
--------------------------------------------------------------------------------
pkt_recv 18988 4970 info packet pktproc Packets received
pkt_recv_zero 18967 4965 info packet pktproc Packets received from QoS 0
pkt_sent 16860 4413 info packet pktproc Packets transmitted
pkt_alloc 19541 5115 info packet resource Packets allocated
session_allocated 50 13 info session resource Sessions allocated
session_freed 18 4 info session resource Sessions freed
session_installed 50 13 info session resource Sessions installed
flow_rcv_dot1q_tag_err 7 1 drop flow parse Packets dropped: 802.1q tag not configured
flow_no_interface 7 1 drop flow parse Packets dropped: invalid interface
flow_ipv6_disabled 12 3 drop flow parse Packets dropped: IPv6 disabled on interface
flow_policy_deny 6 1 drop flow session Session setup: denied by policy
flow_tcp_non_syn 1 0 info flow session Non-SYN TCP packets without session match
flow_tcp_non_syn_drop 1 0 drop flow session Packets dropped: non-SYN TCP without session match
flow_fwd_l3_mcast_drop 9 2 drop flow forward Packets dropped: no route for IP multicast
flow_fwd_zonechange 1 0 drop flow forward Packets dropped: forwarded to different zone
flow_dos_rule_allow_under_rate 1 0 info flow dos Packets allowed: Rate within thresholds of DoS policy
flow_dos_rule_match 1 0 info flow dos Packets matched DoS policy
flow_dos_rule_nomatch 49 12 info flow dos Packets not matched DoS policy
flow_dos_ag_curr_sess_add_incr 1 0 info flow dos Incremented aggregate current session count on session create
flow_dos_cl_curr_sess_add_incr 1 0 info flow dos Incremented classified current session count on session create
flow_dos_ag_buckets_upd 8 2 info flow dos Updated aggregate buckets for aging
flow_arp_pkt_rcv 3 0 info flow arp ARP packets received
flow_arp_pkt_replied 3 0 info flow arp ARP requests replied
flow_host_pkt_rcv 21 5 info flow mgmt Packets received from control plane
flow_host_pkt_xmt 45 11 info flow mgmt Packets transmitted to control plane
flow_host_service_allow 3 0 info flow mgmt Device management session allowed
flow_health_monitor_rcv 19 4 info flow mgmt Health monitoring packet received
flow_health_monitor_xmt 19 4 info flow mgmt Health monitoring packet transmitted
appid_ident_by_icmp 8 2 info appid pktproc Application identified by icmp type
appid_ident_by_dport_first 16 4 info appid pktproc Application identified by L4 dport first
appid_proc 27 7 info appid pktproc The number of packets processed by Application identification
appid_use_dfa_1 12 3 info appid pktproc The number of packets using the second DFA table
appid_unknown_fini_empty 3 0 info appid pktproc The number of unknown applications because of no data
dfa_dte_request_total 5339 1397 info dfa offload The total number of dfa match using dte
dfa_hte_in_cache_lookup 5309 1389 info dfa offload The total number of requests to an in cache HFA graph
tcp_case_2 7 1 info tcp pktproc tcp reassembly case 2
ctd_sml_vm_run_impl_immed8000 19 4 info ctd pktproc SML VM immed8000
ctd_sml_vm_check_domain 16 4 info ctd pktproc sml vm check domain
ctd_sml_opcode_set_file_type 19 4 info ctd pktproc sml opcode set file type
ctd_filter_decode_failure_zip 7 1 error ctd pktproc Number of decode filter failure for zip
ctd_bloom_filter_nohit 16 4 info ctd pktproc The number of no match for virus bloom filter
aho_fpga 10766 2818 info aho resource The total requests to FPGA for AHO
aho_sw 2 0 info aho pktproc The total usage of software for AHO
ctd_appid_reassign 1814 474 info ctd pktproc appid was changed
ctd_pkt_slowpath 5435 1422 info ctd pktproc Packets processed by slowpath
ha_msg_sent 8751 2290 info ha system HA: messages sent
ha_msg_recv 8 2 info ha system HA: messages received
ha_session_setup_msg_sent 45 11 info ha pktproc HA: session setup messages sent
ha_session_teardown_msg_sent 8 2 info ha pktproc HA: session teardown messages sent
ha_session_update_msg_sent 8691 2275 info ha pktproc HA: session update messages sent
ha_arp_update_msg_sent 3 0 info ha pktproc HA: ARP update messages sent
ha_ha2_monitor_msg_sent 8 2 info ha pktproc HA: HA2 monitor message messages sent
ha_ha2_monitor_msg_recv 8 2 info ha pktproc HA: HA2 monitor message messages received
log_url_req_cnt 2 0 info log system Number of url request logs
log_vulnerability_cnt 71 18 info log system Number of vulnerability logs
log_traffic_cnt 23 6 info log system Number of traffic logs
url_db_request 2 0 info url pktproc Number of URL database request
[64;1H [K [7mlines 1-63 [27m [64;1H [64;1H [Kurl_db_reply 2 0 info url pktproc Number of URL reply
zip_process_total 23 6 info zip pktproc The total number of zip engine decompress process
zip_process_failure 7 1 info zip pktproc The number of failures for zip decompress process
pkt_nac_result 10766 2818 info packet resource Packets entered module nac stage result
pkt_flow_np 8200 2146 info packet resource Packets entered module flow stage np
pkt_flow_host 21 5 info packet resource Packets entered module flow stage host
--------------------------------------------------------------------------------
Total counters shown: 63
--------------------------------------------------------------------------------
Global counters:
Elapsed time since last sampling: 4.722 seconds
name value rate severity category aspect description
--------------------------------------------------------------------------------
pkt_recv 18897 4001 info packet pktproc Packets received
pkt_recv_zero 18875 3997 info packet pktproc Packets received from QoS 0
pkt_sent 16528 3500 info packet pktproc Packets transmitted
pkt_alloc 19459 4120 info packet resource Packets allocated
session_allocated 33 6 info session resource Sessions allocated
session_freed 39 8 info session resource Sessions freed
session_installed 33 6 info session resource Sessions installed
session_discard 1 0 info session resource Session set to discard by security policy check
flow_rcv_dot1q_tag_err 9 1 drop flow parse Packets dropped: 802.1q tag not configured
flow_no_interface 9 1 drop flow parse Packets dropped: invalid interface
flow_ipv6_disabled 14 2 drop flow parse Packets dropped: IPv6 disabled on interface
flow_policy_deny 2 0 drop flow session Session setup: denied by policy
flow_fwd_l3_mcast_drop 13 2 drop flow forward Packets dropped: no route for IP multicast
flow_fwd_l3_noarp 3 0 drop flow forward Packets dropped: no ARP
flow_fwd_zonechange 1 0 drop flow forward Packets dropped: forwarded to different zone
flow_parse_unmatched_icmperr 6 1 info flow parse Packets dropped: Unmatched ICMP error message
flow_dos_rule_nomatch 33 6 info flow dos Packets not matched DoS policy
flow_dos_ag_buckets_upd 9 1 info flow dos Updated aggregate buckets for aging
flow_arp_pkt_rcv 4 0 info flow arp ARP packets received
flow_arp_pkt_xmt 1 0 info flow arp ARP packets transmitted
flow_arp_pkt_replied 2 0 info flow arp ARP requests replied
flow_arp_resolve_xmt 1 0 info flow arp ARP resolution packets transmitted
flow_host_pkt_rcv 22 4 info flow mgmt Packets received from control plane
flow_host_pkt_xmt 43 9 info flow mgmt Packets transmitted to control plane
flow_host_service_allow 1 0 info flow mgmt Device management session allowed
flow_health_monitor_rcv 21 4 info flow mgmt Health monitoring packet received
flow_health_monitor_xmt 21 4 info flow mgmt Health monitoring packet transmitted
appid_ident_by_icmp 9 1 info appid pktproc Application identified by icmp type
appid_ident_by_dport_first 5 1 info appid pktproc Application identified by L4 dport first
appid_proc 15 3 info appid pktproc The number of packets processed by Application identification
appid_use_dfa_1 13 2 info appid pktproc The number of packets using the second DFA table
appid_unknown_fini_empty 10 2 info appid pktproc The number of unknown applications because of no data
nat_dynamic_port_xlat 2 0 info nat resource The total number of dynamic_ip_port NAT translate called
dfa_dte_request_total 5384 1140 info dfa offload The total number of dfa match using dte
dfa_hte_in_cache_lookup 5361 1135 info dfa offload The total number of requests to an in cache HFA graph
tcp_case_2 3 0 info tcp pktproc tcp reassembly case 2
ctd_sml_exit_detector_i 1 0 info ctd pktproc The number of sessions with sml exit in detector i
appid_bypass_no_ctd 1 0 info appid pktproc appid bypass due to no ctd
ctd_err_bypass 1 0 info ctd pktproc ctd error bypass
ctd_run_pattern_match_failure 1 0 info ctd pktproc Run pattern match failure
ctd_do_pattern_match 1 0 info ctd pktproc do pattern match
ctd_sml_vm_run_impl_opcodeexit 1 0 info ctd pktproc SML VM opcode exit
ctd_sml_vm_run_impl_immed8000 20 4 info ctd pktproc SML VM immed8000
ctd_sml_vm_check_domain 3 0 info ctd pktproc sml vm check domain
ctd_sml_opcode_set_file_type 21 4 info ctd pktproc sml opcode set file type
ctd_bloom_filter_nohit 3 0 info ctd pktproc The number of no match for virus bloom filter
aho_fpga 10839 2295 info aho resource The total requests to FPGA for AHO
aho_sw 4 0 info aho pktproc The total usage of software for AHO
ctd_appid_reassign 1798 380 info ctd pktproc appid was changed
ctd_url_block 1 0 info ctd pktproc sessions blocked by url filtering
ctd_pkt_slowpath 5447 1153 info ctd pktproc Packets processed by slowpath
ha_msg_sent 8598 1820 info ha system HA: messages sent
ha_msg_recv 9 1 info ha system HA: messages received
ha_session_setup_msg_sent 24 5 info ha pktproc HA: session setup messages sent
ha_session_teardown_msg_sent 7 1 info ha pktproc HA: session teardown messages sent
ha_session_update_msg_sent 8560 1812 info ha pktproc HA: session update messages sent
ha_arp_update_msg_sent 3 0 info ha pktproc HA: ARP update messages sent
[64;1H [K [7mlines 1-63 [27m [64;1H [64;1H [Kha_ha2_monitor_msg_sent 9 1 info ha pktproc HA: HA2 monitor message messages sent
ha_ha2_monitor_msg_recv 9 1 info ha pktproc HA: HA2 monitor message messages received
log_vulnerability_cnt 69 14 info log system Number of vulnerability logs
log_traffic_cnt 33 6 info log system Number of traffic logs
pkt_nac_result 10839 2295 info packet resource Packets entered module nac stage result
pkt_flow_np 8036 1701 info packet resource Packets entered module flow stage np
pkt_flow_host 22 4 info packet resource Packets entered module flow stage host
--------------------------------------------------------------------------------
Total counters shown: 64
--------------------------------------------------------------------------------
Global counters:
Elapsed time since last sampling: 3.18 seconds
name value rate severity category aspect description
--------------------------------------------------------------------------------
pkt_recv 1006 333 info packet pktproc Packets received
pkt_recv_zero 994 329 info packet pktproc Packets received from QoS 0
pkt_sent 1832 607 info packet pktproc Packets transmitted
pkt_alloc 1075 356 info packet resource Packets allocated
session_allocated 13 4 info session resource Sessions allocated
session_freed 15 4 info session resource Sessions freed
session_installed 13 4 info session resource Sessions installed
flow_rcv_dot1q_tag_err 4 1 drop flow parse Packets dropped: 802.1q tag not configured
flow_no_interface 4 1 drop flow parse Packets dropped: invalid interface
flow_ipv6_disabled 6 1 drop flow parse Packets dropped: IPv6 disabled on interface
flow_policy_deny 3 0 drop flow session Session setup: denied by policy
flow_fwd_l3_mcast_drop 9 2 drop flow forward Packets dropped: no route for IP multicast
flow_fwd_l3_noarp 1 0 drop flow forward Packets dropped: no ARP
flow_parse_unmatched_icmperr 1 0 info flow parse Packets dropped: Unmatched ICMP error message
flow_dos_rule_allow_under_rate 1 0 info flow dos Packets allowed: Rate within thresholds of DoS policy
flow_dos_rule_match 1 0 info flow dos Packets matched DoS policy
flow_dos_rule_nomatch 12 3 info flow dos Packets not matched DoS policy
flow_dos_ag_curr_sess_add_incr 1 0 info flow dos Incremented aggregate current session count on session create
flow_dos_cl_curr_sess_add_incr 1 0 info flow dos Incremented classified current session count on session create
flow_dos_ag_buckets_upd 6 1 info flow dos Updated aggregate buckets for aging
flow_arp_pkt_rcv 3 0 info flow arp ARP packets received
flow_arp_pkt_replied 1 0 info flow arp ARP requests replied
flow_arp_rcv_gratuitous 1 0 info flow arp Gratuitous ARP packets received
flow_host_pkt_rcv 12 3 info flow mgmt Packets received from control plane
flow_host_pkt_xmt 19 6 info flow mgmt Packets transmitted to control plane
flow_host_service_allow 2 0 info flow mgmt Device management session allowed
flow_health_monitor_rcv 11 3 info flow mgmt Health monitoring packet received
flow_health_monitor_xmt 11 3 info flow mgmt Health monitoring packet transmitted
appid_ident_by_icmp 4 1 info appid pktproc Application identified by icmp type
appid_proc 9 2 info appid pktproc The number of packets processed by Application identification
appid_use_dfa_1 7 2 info appid pktproc The number of packets using the second DFA table
dfa_dte_request_total 106 35 info dfa offload The total number of dfa match using dte
dfa_hte_in_cache_lookup 99 32 info dfa offload The total number of requests to an in cache HFA graph
ctd_sml_vm_run_impl_immed8000 12 3 info ctd pktproc SML VM immed8000
ctd_sml_opcode_set_file_type 15 4 info ctd pktproc sml opcode set file type
aho_fpga 85 28 info aho resource The total requests to FPGA for AHO
ctd_pkt_slowpath 102 33 info ctd pktproc Packets processed by slowpath
ha_msg_sent 984 326 info ha system HA: messages sent
ha_msg_recv 6 1 info ha system HA: messages received
ha_session_setup_msg_sent 11 3 info ha pktproc HA: session setup messages sent
ha_session_teardown_msg_sent 7 2 info ha pktproc HA: session teardown messages sent
ha_session_update_msg_sent 962 318 info ha pktproc HA: session update messages sent
ha_arp_update_msg_sent 1 0 info ha pktproc HA: ARP update messages sent
ha_ha2_monitor_msg_sent 6 1 info ha pktproc HA: HA2 monitor message messages sent
ha_ha2_monitor_msg_recv 6 1 info ha pktproc HA: HA2 monitor message messages received
log_traffic_cnt 21 6 info log system Number of traffic logs
pkt_nac_result 85 28 info packet resource Packets entered module nac stage result
pkt_flow_np 909 301 info packet resource Packets entered module flow stage np
pkt_flow_host 12 3 info packet resource Packets entered module flow stage host
--------------------------------------------------------------------------------
Total counters shown: 49
--------------------------------------------------------------------------------
07-08-2014 09:03 AM
Thanks for your input!
In this firewall, we do not have any rules which have an application specified along with service ANY. We do however have several rules which list an application with service "application-default". Is it faster processing wise to explicitly list the port as oppose to app default?
07-08-2014 09:03 AM
Hello Mackwage,
Output looks good, no issue. Would it be possible to provide packet captures.
Regards,
Hardik Shah
07-08-2014 09:06 AM
Will get those to you shortly. One question on the buffer output...
Hardware Pools
[ 0] Packet Buffers : 11341/11468 0x8000000031000000
[ 1] Work Queue Entries : 229309/229376 0x8000000037ffe000
[ 2] Output Buffers : 1006/1024 0x8000000039bfe000
[ 3] DFA Result : 3999/4000 0x8000000039cfe000
[ 4] Timer Buffers : 4092/4096 0x800000003a0e6000
[ 5] PAN_FPA_LWM_POOL : 1024/1024 0x8000000000e9f200
[ 6] PAN_FPA_ZIP_POOL : 1023/1024 0x800000003a4e6000
[ 7] PAN_FPA_BLAST_POOL : 1024/1024 0x800000000ff00000
Software Pools
[ 0] software packet buffer 0 : 32767/32768 0x800000003a6e6680
[ 1] software packet buffer 1 : 32768/32768 0x800000003b706700
[ 2] software packet buffer 2 : 81920/81920 0x800000003d726780
[ 3] software packet buffer 3 : 20480/20480 0x8000000047776800
The buffers say for example 32767/32768. Does this imply the buffer is almost full? Or is it specifying it is empty?
07-08-2014 09:07 AM
Buffers are empty, buffers are not utilized, buffers are under utilized.
07-08-2014 09:18 AM
There are some drop counters..
flow_policy_deny 6 1 drop flow session Session setup: denied by policy
flow_tcp_non_syn 1 0 info flow session Non-SYN TCP packets without session match
flow_tcp_non_syn_drop 1 0 drop flow session Packets dropped: non-SYN TCP without session match
flow_fwd_l3_mcast_drop 9 2 drop flow forward Packets dropped: no route for IP multicast
flow_fwd_zonechange 1 0 drop flow forward Packets dropped: forwarded to different zone
Could you configure a packet-filter based on the source and destination used for testing and collect new counters?
> debug dataplane packet-diag set filter match source x.x.x.x destination y.y.y.y
> debug dataplane packet-diag set filter on
Then collect counters with packet-filter
> show counter global name filter packet-filter yes delta yes
*Note* the filter will only match new sessions, not sessions already present in session table
07-08-2014 09:23 AM
I have the pcaps but I am not seeing a forum option to upload attachments. :smileylaugh:
07-08-2014 09:25 AM
mackwage wrote:
Thanks for your input!
In this firewall, we do not have any rules which have an application specified along with service ANY. We do however have several rules which list an application with service "application-default". Is it faster processing wise to explicitly list the port as oppose to app default?
I don't have any direct experience with that situation. I would assume that it would only affect your target traffic if one of the default application ports overlapped with your delayed traffic.
Click Accept as Solution to acknowledge that the answer to your question has been provided.
The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!
These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!
The LIVEcommunity thanks you for your participation!
