Routing Between Virtual Routers in Same Firewall

[Jama]

I have two Virtual routers in same Firewall I wanted to allow traffic between the Virtual routers, I configured  rules to allow traffic from Trusted L3 zone in VR1 to Trusted zone in VR2 and vice-verse and put them at the top of the rules and also I configured static routes between VR's.

The ICMP is working fine I can ping all of network from one VR to another VR but web-accessing isn't working, although the source/destination application is any

I monitored and the application section is incomplete.

With same VR everything is working even HTTP/HTTPS access but the moment I try to access the other VR network it's not working.

PA-5050

Version: 5.0.2

Pls Help me.

Jama Yassin

[UhMayYeah]

Try the troubleshooting following from the CLI:

> debug dataplane packet-diag clear all

> debug dataplane packet-diag set filter match source <ip> destination <ip>

> debug dataplane packet-diag set filter on

Initiate the traffic between the client and server,Run this command multiple times and watch for any drop or warn counters incrementing .

> show counter global filter packet-filter yes delta yes severity warn

-Ameya

[apasupulati]

Hey Jama,

Application incomplete means that the TCP 3-way handshake was unsuccessful. So to explain a little clearer, if a client sends a server a syn and the paloalto device creates a session for that syn, but the server never sends a syn ack in response back to the client, then that session would be seen as incomplete.

So you can configure packet captures on the device and look for the complete TCP handshake. Make sure client to server and server to client communication is good.

Refer to this article to know how to configure filters and capture the traffic,

Hope that helps.

Aditi

[Jama]

Ameya/Aditi

I have run packet capture in CLI.

This is the outcome.

Name                                   value     rate severity  category  aspect    description

--------------------------------------------------------------------------------

session_install_error                   1064        7 warn      session   pktproc   Sessions installation error

session_inter_cpu_install_error            4        0 warn      session   pktproc   Inter-CPU Session installation error

session_inter_cpu_sync_err               359        2 warn      session   resource  Inter-DP packet does not match a session

flow_ipv6_disabled                        25        0 drop      flow      parse     Packets dropped: IPv6 disabled on interface

flow_policy_deny                       14496       96 drop      flow      session   Session setup: denied by policy

flow_policy_nat_land                     322        2 drop      flow      session   Session setup: source NAT IP allocation result in LAND attack

flow_tcp_non_syn_drop                   4766       31 drop      flow      session   Packets dropped: non-SYN TCP without session match

flow_fwd_l3_mcast_drop                    14        0 drop      flow      forward   Packets dropped: no route for IP multicast

flow_fwd_l3_ttl_zero                       3        0 drop      flow      forward   Packets dropped: IP TTL reaches zero

flow_fwd_l3_noarp                         18        0 drop      flow      forward   Packets dropped: no ARP

flow_predict_reused                       24        0 warn      flow      pktproc   Predict session starts before parent, possible reuse case

flow_action_close                        622        3 drop      flow      pktproc   TCP sessions closed via injecting RST

flow_host_service_deny                   416        2 drop      flow      mgmt      Device management session denied

flow_host_service_unknown              11284       75 drop      flow      mgmt      Session discarded: unknown application to control plane

appid_lookup_invalid_flow                 80        0 drop      appid     pktproc   Packets dropped: invalid session state

tcp_bypass                                10        0 warn      tcp       pktproc   session skip L7 proc because of failure in tcp reassembly

tcp_drop_packet                          166        0 warn      tcp       pktproc   packets dropped because of failure in tcp reassembly

tcp_out_of_sync                           29        0 warn      tcp       pktproc   can't continue tcp reassembly because it is out of sync

tcp_drop_out_of_wnd                       45        0 warn      tcp       resource  out-of-window packets dropped

tcp_exceed_flow_seg_limit                  8        0 warn      tcp       resource  packets dropped due to the limitation on tcp out-of-order queue siz

e

tcp_new_syn                               50        0 warn      tcp       pktproc   A new SYN packet in tcp session

ctd_file_forward_error                     4        0 error     ctd       pktproc   The number of file forward error found

ctd_filter_decode_failure_zip           3574       22 error     ctd       pktproc   Number of decode filter failure for zip

ctd_skip_offset_error                      3        0 warn      ctd       resource  skip offset error

url_request_pkt_drop                   10262       66 drop      url       pktproc   The number of packets get dropped because of waiting for url catego

ry request

--------------------------------------------------------------------------------

Total counters shown: 25

--------------------------------------------------------------------------------

flow_tcp_non_syn_drop                      2        0 drop      flow      session   Packets dropped: non-SYN TCP without session match

flow_tcp_non_syn_drop                     12        0 drop      flow      session   Packets dropped: non-SYN TCP without session match

Jama

[apasupulati]

Hey Jama,

Did you have packet filters configured while you collected these counters?

To deal with the non-SYN tcp drops, can you the run following command from CLI and see if that helps with the inter-VR communication:

> set session tcp-reject-non-syn no

Note: this command isn't persistent through a commit/reboot.

The firewall by default rejects any non-SYN packets (SYN-ACK, ACK) that don't match an existing session, we can disable this feature for testing and see if that helps. The reason the packets don't match the existing session could be that the response took too long and the session expired OR that the packets return from a different zone/interface causing asymmetric routing.

Let me know.

Aditi

[Jama]

Aditi.

Yes I configured the packet filtering from CLI.

I put that command from CLI but there is no effect it's still same.

I think I have forgot to tell you something I found your message is that some times I can see that the server is responded and even can get the login screen. But it takes at least 30-40 minutes to get respond.

May be you are right the packets is returning from a different zone/interface is causing asymmetric routing.

How can I solve it?..

[UhMayYeah]

try these settings

# set deviceconfig setting tcp asymmetric-path bypass

# set deviceconfig setting session tcp-reject-non-syn no

# commit

P.S:These commands would bypass important TCP inspections.

Verify packet filter setting:

> debug dataplane packet-diag show setting

Initiate the traffic between the client and server,Run this command multiple times and watch for any drop or warn counters incrementing .

> show counter global filter packet-filter yes delta yes severity warn

[Jama]

I have followed your instructions but it's still same outcome

This is the outcome of CLI packet filtering

name

value

rate severity  category  aspect

description

--------------------------------------------------------------------------------

session_install_error	182	20 warn	session   pktproc   Sessions installation error
session_inter_cpu_sync_err	100	11 warn	session   resource  Inter-DP packet does not match a session
flow_ipv6_disabled	6	0 drop	flow	parse	Packets dropped: IPv6 disabled on interface
flow_policy_deny	4110	462 drop	flow	session   Session setup: denied by policy
flow_policy_nat_land	76	8 drop	flow	session   Session setup: source NAT IP allocation result in LAND attack
flow_fwd_l3_mcast_drop	2	0 drop	flow	forward   Packets dropped: no route for IP multicast
flow_fwd_zonechange	20	1 drop	flow	forward   Packets dropped: forwarded to different zone
flow_predict_reused	30	3 warn	flow	pktproc   Predict session starts before parent, possible reuse case
flow_action_close	14	1 drop	flow	pktproc   TCP sessions closed via injecting RST
flow_host_service_deny	64	7 drop	flow	mgmt	Device management session denied
flow_host_service_unknown	1235	138 drop	flow	mgmt	Session discarded: unknown application to control plane
appid_lookup_invalid_flow	14	1 drop	appid	pktproc   Packets dropped: invalid session state
tcp_bypass	7	0 warn	tcp	pktproc   session skip L7 proc because of failure in tcp reassembly
tcp_drop_packet	9	0 warn	tcp	pktproc   packets dropped because of failure in tcp reassembly
tcp_out_of_sync	2	0 warn	tcp	pktproc   can't continue tcp reassembly because it is out of sync
tcp_exceed_flow_seg_limit	5	0 warn	tcp	resource  packets dropped due to the limitation on tcp out-of-order queue size
tcp_new_syn	14	1 warn	tcp	pktproc   A new SYN packet in tcp session
ctd_filter_decode_failure_zip	578	64 error	ctd	pktproc   Number of decode filter failure for zip
ctd_skip_offset_error	3	0 warn	ctd	resource  skip offset error
url_request_pkt_drop	1555	174 drop	url	pktproc   The number of packets get dropped because of waiting for url category req

[Naresh41]

Hi,

can you please share the document for Routing Between Virtual Routers in Same Firewall. i need to setup same in our enviorment.

Regards

Naresh Kumar