Panorama connectivity check failed for xxxx. Reason: TCP channel setup failed, reverting configuration

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Announcements

Panorama connectivity check failed for xxxx. Reason: TCP channel setup failed, reverting configuration

L2 Linker

- We ran into an issue where the commits from Panorama were failing with error:

• . Performing panorama connectivity check (attempt 1 of 1)
• . Panorama connectivity check failed for xxxx. Reason: TCP channel setup failed, reverting configuration
• . Configuration reverted successfully

- We checked No validation errors while the commit failed
- run show panorama-status on the managed firewall
> show panorama-status
Panorama Server 1 : xxxx
Connected : yes
HA state : Unknown

- Check if Pings between the Firewalls and Panorama are working

> ping host x.x.x.x  - 100 success

- Check if Netstat output on the Firewalls show connnections are Established to the Panorama on port 3978.
> show netstat all yes numeric-hosts yes numeric-ports yes
tcp  x.x.x.x:3978 ESTABLISHED

- Connectivity looks fine, tried restarting the management server process
> debug software restart process management-server
- We tried to add the ip address instead of the hostname - but it would still fail
- We tried to push the Device-group for the other cluster as well and it fails with the same reason
- We see show system resources – pointing at 3 zombie processes in the panorama
- We see system files are present 
- Analyzing the TSF uploaded – we see
2 (2023-09-19 11:07:19) – Critical System(java) - An Out of Memory condition occured and restarted the process
Sep 19 11:04:43 CMS init: Id "s0" respawning too fast: disabled for 5 minutes
Sep 19 11:07:18 CMS klogd: conn12 invoked oom-killer: gfp_mask=0x201da, order=0, oom_score_adj=0
Sep 19 11:07:19 CMS klogd: Out of memory: Kill process 3399 (java) score 504 or sacrifice child
Sep 19 11:07:19 CMS klogd: Killed process 3399 (java) total-vm:686690928kB, anon-rss:8422608kB, file-rss:0kB
We also see
2 (2023-09-13 23:31:25)- Critical System (nodejs ) – Process Restarts - nodejs restarted unexpectedly however no known issue was identified, and we see the java core files which explains the restarts.
- Reboot didn't make a change, we still saw the OOM errors
- Lastly also upgraded the Pano to : 10.1.10-h2 but the issue persists

 

1 accepted solution

Accepted Solutions

L2 Linker

We raised a TAC case and they stated there is an internal investigation still going on this.

Resolution- increase the Number of attempts to check for Panorama connectivity to 5

.

View solution in original post

1 REPLY 1

L2 Linker

We raised a TAC case and they stated there is an internal investigation still going on this.

Resolution- increase the Number of attempts to check for Panorama connectivity to 5

.

  • 1 accepted solution
  • 10597 Views
  • 1 replies
  • 0 Likes
Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!

The LIVEcommunity thanks you for your participation!