Reason: TCP channel setup failed, reverting configuration issue.

[zGomez]

Hello,

Since recently we have a few firewalls that we are unable to push because the firewall is checking connectivity to panorama and this is failing. 

Inside panorama the device is listed as connected and from the firewall's session table I can see there is an existing session to panorama.

2023-06-07 16:38:38.410 +0200 ACR: Performing panorama connectivity check (attempt 5 of 5)
2023-06-07 16:38:38.410 +0200 [Secure conn] Secure channel for Firewall to panorama communication not enabled for secure conn.
2023-06-07 16:38:56.329 +0200 client dagger reported op command was SUCCESSFUL
2023-06-07 16:38:57.459 +0200 client dagger reported op command was SUCCESSFUL
2023-06-07 16:38:58.807 +0200 Error: pan_comm_get_iplist(cs_conn.c:4711): connmgr: panorama: addr info address: panorama.domain.net error: System error
2023-06-07 16:38:58.808 +0200 Error: pan_cmsa_tcp_channel_setup(src_panos/cms_agent.c:1124): ACR: Failed to establish TCP connection
2023-06-07 16:38:58.808 +0200 ACR: Panorama connectivity check failed for panorama.ontex.net. Reason: TCP channel setup failed, reverting configuration
2023-06-07 16:38:58.808 +0200 ACR: Post-commit connectivity check failed, beginning to revert config.

I already tried increasing timers and amount of retries.   I also verified the firewall is able to reach panorama and is connected.

DNS is working.

Session table is showing me 2 active sessions to panorama.

show session all filter destination 10.255.125.50

--------------------------------------------------------------------------------
ID Application State Type Flag Src[Sport]/Zone/Proto (translated IP[Port])
Vsys Dst[Dport]/Zone (translated IP[Port])
--------------------------------------------------------------------------------
6501 panorama ACTIVE FLOW 10.163.66.253[33607]/management/6 (10.163.66.253[33607])
vsys1 10.255.125.50[3978]/VPN (10.255.125.50[3978])
7007 panorama ACTIVE FLOW 10.163.66.252[45224]/management/6 (10.163.66.252[45224])
vsys1 10.255.125.50[3978]/VPN (10.255.125.50[3978])

anybody else experiencing this?    can i use global counter for management traffic?

Only one of the firewalls in the cluster is having this issue, only active one.  Restarting mangement plane did not help.

[Raido_Rattameister · Accepted solution]

If you change Panorama from DNS name to IP it still fails?

[Jeroen_Proost]

@zGomez Have you found a solution yet ?
I have a similar problem, but unfortunately no solution yet.

[zGomez]

Hi Jeroen,

For met the issue was resolved by checking the primary and secondary dns used under setup, services.  The primary dns in use here was an old dns that was no longer responding.  when issuing a dns lookup from the cli of palo alto i always had a response from the mgt interface.  So i am guessing the panorama check never switches to the seconcary if first is not responding.

Dns resolving was something i checked right away fromt he cli but since this was responding i did not immediatly check the services dns config.

I tried first as Raido suggest the ip and then it worked so this made me look at dns settings.

[bwsaloum]

I started having this same issue while attempting to add a second vpn tunnel to a 220.  The moment I start seeing that message in the firewall system logs, it appears to drop offline in Panorama.  Weird thing is that I can still https and ssh to it...  About the only way  I've been able to recover is to restart the management-server and eventually it reconnects.

[ChuckW]

I too, am experiencing this issue and Panorama has always been referenced by IP and not DNS name.

I am trying to enable ECMP on a HA pair PA5260s

[karldormeTVH]

Hi,
I observe the same, which version are you running ? I'm on 10.1.10-h2

[AtulK]

I have had the same issue with a virtual firewall managed by Panorama. I do not use the hostname, but connect using IP address. Initially, PANW TAC suggested adding the IP address of Panorama on the managed firewall under Device > Setup > Interfaces > Management > Permitted IP Addresses. The issue was resolved for a while just after making this change, however it has re-appeared. I *think* my issue is to do with the server infrastructure that this virtual firewalls sits on, but have no concrete proof, so I have logged a TAC case again. 

Versions running:
Panorama: 10.2.3-h2

Managed firewall: 10.1.6

[Jeroen_Proost]

The solution in my case was not only to factory reset the PA440, but also delete every remaining default configuration in it. After that, there was no problem pushing config to the PA440's.

[J-Miller]

Updating the DNS (as you noted) correct this issue for me.  Thank you for the post.

[AtulK]

Just adding further to this for folks who might have issues in the future. My issue was resolved by increasing the "number of attempts for Panorama connectivity" from 1 (default) to 5. As I understand it, this solution does not address the underlying cause for this issue, which I *think* could be the bandwidth limitation since Panorama resides in Azure and the firewall is in a DC.

I followed this article to make changes:
https://docs.paloaltonetworks.com/pan-os/9-1/pan-os-new-features/panorama-features/automatic-panoram...

[D_Milojevic]

Not sure if you located answer to your issue but in my case (same error) i had to install certificate on my firewall and after that no error...

So far...

Cheers

[Ernesto-James]

Came across the same issue, but the only X-factors were updated to permitted IP's for the MGT interfaces.  Had to Add the firewall IP addresses to the Permitted IP's for the MGT interfaces on Panorama.

[jmatanane]

 This broke my connection to the GUI of those firewalls that I added the permit IP address. Not advised