Add source NAT: Bi-directional NAT breaks

[cstech]

Hello,

1. I have three zones: Public, DMZ, Private

Public interface: bound to 164.67.80.124/26

Private interface: bound to 192.168.1.1/24

DMZ interface: bound to 192.168.2.1/24

2. I use bi-directional NAT to expose a few of the DMZ machines via their own public IP addresses:

164.67.80.77 <-> 192.168.2.77

164.67.80.78 <-> 192.168.2.78

164.67.80.79 <-> 192.168.2.79

Example:

Source zone: DMZ

Destination zone: Public

Destination Interface: any

Source Address: 192.168.2.77

Destination Address: any

Service: any

Source Translation: static-ip, 164.67.80.77, bi-directional: yes

Destination Translation: none

3. I add security policies to allow the DMZ machines to access the internet and be accessed from the internet.

4. None of my interfaces are explicitly bound to 164.67.80.77, 78, or 79.  This works due to something called "Proxy ARP" on the public interface (as explained to me in these discussion forums).

5. I add a "Source NAT" policy with the intention of providing internet access to the Private zone:

Source zone: Private

Destination zone: Public

Destination Interface: any

Source Address: any

Destination Address: any

Service: any

Source Translation: dynamic-ip-and-port, 164.67.80.124

Destination Translation: none

6. I don't yet add a security policy to allow Private machines to the Public zone.

7. After step 5, the communication between the DMZ and Public stops.  Internet cannot access the DMZ machines via the public IP addresses and the DMZ cannot access the internet.

Question: Why does the DMZ stop working when I perform step 5?

Thank you,

Chris

[reaper]

Hi Chris

The public IP's of your dmz servers are also identified by the unit as being in the "public" zone, hence your source NAT policy may be taking over any previous nat/no-nat actions.

eg: 192.16.1.10 to 164.67.80.77 equals trust to untrust in the sense of NAT translation

The bi directional rules will have a second implied rule for the inbound traffic "from any ; to untrust ; to ip 164.67.80.x ; translate to 192.16.2.x", allowing both external and internal hosts to NAT to the proper dmz IP

I would guess the new source nat rule is at the top of your policy? if you move it down to the bottom and commit the issue should be resolved

[cstech]

tpiens,

Thank you for your reply.

Re your comment "192.16.1.10 to 164.67.80.77 equals trust to untrust in the sense of NAT translation" thank you I would have been unsure of that if/when I thought about it.

Re the second implied rule for the bi-directional NAT policy, I figured something like that must be happening.

Re the position of the source NAT rule that breaks my DMZ: Unfortunately the new source NAT rule is and has always been at the bottom of the list.

Is there anything is unusual about my setup (I thought it was straight forward)?  If nothing is unusual about my setup, then I will have to look for something really odd on my network like a cable plugged into the wrong port etc.

Thank you again,

Chris

[reaper]

Hi Chris

The setup in itself is straight forward but there can always be nuances missing, would you mind checking your sessions, there should be an indication of what NAT is being applied by looking at the c2s/s2c flows

I usually recommend configuring the bidirectional NAT rules as 2 separate rules as this allows a little more visibility in what has been configured exactly. Are you accessing your DMZ servers on their physical IP's or on the public ones ?

[cstech]

tpiens,

Thank you again for your help.  I am embarrassed to say that I have never "checked my sessions" but I am going to do that now.  I will split the bi-directional NAT policy and hopefully in the process will discover my error.

Chris

[reaper]

HI Chris

Don't sweat it, that means everything's been going smoothly

the sessions can be seen either through the session browser under the monitor tab in the GUI, or through the >show session all / >show session id ## commands in cli and look sort of like this:

admin@PA-3020-241> show  session  id 32002

Session           32002

        c2s flow:

                source:      192.168.0.21 [v1-trust]

                dst:         4.2.2.2

                proto:       1

                sport:       1024            dport:      15104

                state:       INIT            type:       FLOW

                src user:    unknown

                dst user:    unknown

        s2c flow:

                source:      4.2.2.2 [v1-untrust]

                dst:         172.16.31.241

                proto:       1

                sport:       15104           dport:      1024

                state:       INIT            type:       FLOW

                src user:    unknown

                dst user:    unknown

in the c2s you see your original ip's as the client tries to reach them, in the s2c you see the expected returning ip's. in a normal hide-NAT setup only the dst would be different as this is now an external ip in the firewall untrust interface

In your case I'd expect the dst would need to be an internal IP and source the dmz server physical ip. if this is not the case one of your nat rules is doing something unexpected

further down the session details you can see exactly which NAT rule is being hit:

start time                    : Wed Sep  3 00:24:19 2014

        timeout                       : 6 sec

        total byte count(c2s)         : 74

        total byte count(s2c)         : 74

        layer7 packet count(c2s)      : 1

        layer7 packet count(s2c)      : 1

        vsys                          : vsys1

        application                   : ping

        rule                          : out

        session to be logged at end   : True

        session in session ager       : False

        session synced from HA peer   : False

        address/port translation      : source + destination

        nat-rule                      : hidenat(vsys1)

        layer7 processing             : enabled

        URL filtering enabled         : False

        session via syn-cookies       : False

        session terminated on host    : False

        session traverses tunnel      : False

        captive portal session        : False

        ingress interface             : ethernet1/2

        egress interface              : ethernet1/1

        session QoS rule              : N/A (class 4)

        tracker stage firewall        : Aged out

hope this helps

Tom

[cstech]

I do not know why it's happening, but what I believe should be Public->Public traffic is being dropped.  I can see the data is dropped by the PAN by inspecting the drop log from the packet capture.

Eager to understand why the traffic is dropped, I added four security policies: Allow public->public, allow DMZ->DMZ, allow Private->Private, Deny All->All.  Every one of my security policies has logging turned on.

Would you believe that the PAN is dropping packets (as evidenced in the packet capture drop log) but there is no trace of it in the Traffic Log?

I have seen this before: it happens for example when a PAN interface drops ping due to the management profile.  Not sure why it is happening in my case though.

Besides adding a "catch all" deny all->all, is there another way to get the drops to appear in the traffic log?

Thank you,

Chris

[cstech]

BTW, there is no session log for this traffic... presumably because no session can be established.

[cstech]

Another way traffic can be dropped but not appear in the traffic log is if the next hop is unavailable...

[reaper]

Hi Chris

you could leverage packet-diag filters and global counters to get an indication of what's happening as well, you can also combine with pcaps to see what is coming into and what is leaving the firewall

Packet Capture, Debug Flow-basic and Counter Commands

it sounds like you may be getting a zone issue or LAND attack due to NAT (source nat an internal ip to an untrust interface ip with destination untrust interface ip) and counters may help shed light

It may help adding specific nat rules for the dmz public addresses from source zone trust

so if you go ahead and split up your nat rules there would be 3 sets per server:

untrust to untrust; any to public ip; destination nat to physical ip

trust to untrust; any to public ip; destination nat to physical ip 

            (this is a half-U-turn which is pretty common if you're communicating to internal resources via their public ip  How to Configure U-Turn NAT)

dmz to untrust; physical ip to any; source nat to public ip

and then at the end your hide nat

trust to untrust; any to any; source nat to interface ip

[cstech · Accepted solution]

Thank you Tom for your help with this.  I solved the problem.  In my original post I described my source NAT rule's source translation:

"Source Translation: dynamic-ip-and-port, 164.67.80.124"

What I neglected to include in the post was the subnet mask.  I used the same address object in the NAT rule that I used for the interface: 164.67.80.124/26.  This was a mistake.  I should have created a second address object for the IP address with a subnet mask of /32: 164.67.80.124/32.

This doesn't explain all the problems I described in this post.  And unfortunately for me (or fortunately depending on your viewpoint) some of them have gone away so I cannot examine them anymore.

Anyhow, thank you again for your advice and introducing me to the LAND attack which is a source of packet drops that show up only in the counters and not in the logs.  And thank you for pointing out that I can can more fine grained control over my NAT rules by splitting the bi-directional rules.

Chris