Disconnected from Log collector Server

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Announcements

Disconnected from Log collector Server

Cyber Elite
Cyber Elite

 

Tonight we got email alerts that our firewalls are disonncted from the log collecors-M500

 

Below is ms log from the PA

 

2019-04-05 01:38:55.024 -0600 MS: disconnected from log-collector. waitcount=1
2019-04-05 01:38:55.024 -0600 lcs agent: channel teardown (to 10.7.1.139) complete.
2019-04-05 01:38:55.035 -0600 Error: pan_conn_ext_send_base(cs_conn.c:2601): connmgr: send failure. no conn entry: devid=log-collector
2019-04-05 01:38:55.035 -0600 Error: pan_cfg_log_buffer_nsend(pan_cfg_log_buffer.c:172): Failed to send the logrec to log collector
2019-04-05 01:38:55.035 -0600 Error: pan_log_buffer_cursor_next(pan_cfg_log_buffer.c:1538): logbuffer: failed to send 10 'system' logs to cms
2019-04-05 01:39:00.025 -0600 COMM: connection established. sock=50 remote ip=10.7.1.139 port=3978 local port=47854
2019-04-05 01:39:00.025 -0600 lcs agent: Pre. send buffer limit=22600. s=50
2019-04-05 01:39:00.025 -0600 lcs agent: Post. send buffer limit=1048576. s=50
2019-04-05 01:39:02.057 -0600 lcs agent: ssl channel established. sock=50 ssl=0x121a3400
2019-04-05 01:39:02.058 -0600 Error: pan_mgmt_get_sysd_string(pan_cfg_status_handler.c:372): failed to fetch cfg.saas.custid
2019-04-05 01:39:02.059 -0600 Error: pan_get_current_gp_datafile_release_date(pan_cfg_utils.c:5526): Failed to parse file /opt/pancfg/mgmt/global-protect/av-data/av_data_file.dat
2019-04-05 01:39:02.179 -0600 lcs agent: registration request sent. len=29215 sock=50
2019-04-05 01:39:02.260 -0600 connmgr: connection entry added: devid=log-collector sock=50, clientid=0
2019-04-05 01:39:02.261 -0600 connected to Log Collector 007307001117 (key log-collector)
2019-04-05 01:39:02.278 -0600 received a log-fwd-ctrl(start-from-lastack) message from panorama
2019-04-05 01:39:02.278 -0600 Logbuffer: start_from[0] LastAck
2019-04-05 01:39:02.278 -0600 Logbuffer: start_from[1] LastAck
2019-04-05 01:39:02.278 -0600 Logbuffer: start_from[2] LastAck
2019-04-05 01:39:02.278 -0600 Logbuffer: start_from[3] LastAck
2019-04-05 01:39:02.278 -0600 Logbuffer: start_from[4] LastAck
2019-04-05 01:39:02.278 -0600 Logbuffer: start_from[5] LastAck
2019-04-05 01:39:02.278 -0600 Logbuffer: start_from[6] LastAck
2019-04-05 01:39:02.278 -0600 Logbuffer: start_from[7] LastAck
2019-04-05 01:39:02.278 -0600 Logbuffer: start_from[8] LastAck
2019-04-05 01:39:02.278 -0600 Logbuffer: start_from[9] LastAck
2019-04-05 01:39:02.278 -0600 Logbuffer: start_from[10] LastAck
2019-04-05 01:39:02.278 -0600 Logbuffer: start_from[11] LastAck
2019-04-05 01:39:02.278 -0600 Logbuffer: start_from[12] LastAck
2019-04-05 01:39:02.278 -0600 Logbuffer: start_from[13] LastAck
2019-04-05 01:39:02.278 -0600 Logbuffer: start_from[14] LastAck
2019-04-05 01:39:02.278 -0600 Logbuffer: start_from[15] LastAck
2019-04-05 01:39:02.278 -0600 Logbuffer: start_from[16] LastAck
2019-04-05 01:39:02.278 -0600 Logbuffer: start_from[17] LastAck
2019-04-05 01:39:02.278 -0600 Logbuffer: start_from[18] LastAck
2019-04-05 01:39:02.278 -0600 Logbuffer: start_from[19] LastAck
2019-04-05 01:39:02.484 -0600 Error: pan_log_query_expr_get_time_envelope(pan_logdb.c:2598): failed to determine time envelope
2019-04-05 01:39:02.484 -0600 latest logid for hipmatch: 0, recvtime: 0
2019-04-05 01:39:02.484 -0600 Error: pan_log_query_expr_get_time_envelope(pan_logdb.c:2598): failed to determine time envelope
2019-04-05 01:39:21.651 -0600 latest logid for userid: 0, recvtime: 0
2019-04-05 01:39:21.652 -0600 Error: pan_log_query_expr_get_time_envelope(pan_logdb.c:2598): failed to determine time envelope
2019-04-05 01:39:21.652 -0600 latest logid for gtp: 0, recvtime: 0
2019-04-05 01:39:21.653 -0600 Error: pan_log_query_expr_get_time_envelope(pan_logdb.c:2598): failed to determine time envelope
2019-04-05 01:39:21.653 -0600 latest logid for auth: 0, recvtime: 0
2019-04-05 01:40:07.274 -0600 Error: pan_conn_entry_write_lock_timeout(cs_conn.c:4227): connmgr: get wr lock timeout. result=110
2019-04-05 01:40:42.276 -0600 Error: pan_conn_entry_write_lock_timeout(cs_conn.c:4227): connmgr: get wr lock timeout. result=110
2019-04-05 01:41:17.282 -0600 Error: pan_conn_entry_write_lock_timeout(cs_conn.c:4227): connmgr: get wr lock timeout. result=110
2019-04-05 01:41:52.285 -0600 Error: pan_conn_entry_write_lock_timeout(cs_conn.c:4227): connmgr: get wr lock timeout. result=110
2019-04-05 01:42:22.024 -0600 Error: csSendWithTimeoutChunk(cs_comm_utils.c:443): COMMS: sock=50. SSL write error fatal. code=5 error=Broken pipe(32) retrycount=0 len=8052 remain=8052 sent=-1
2019-04-05 01:42:22.024 -0600 Error: cs_msg_tcp_send_ex(cs_transport.c:165): COMM: failed to send payload. result=0 len=8052 ctype=3 dtype=1 mtype=0 sock=50 ssl=0x121a3400
2019-04-05 01:42:22.024 -0600 Device lcs agent log-collector disconnected
2019-04-05 01:42:22.024 -0600 connmgr: shutdown channel. sock=50 ssl=0x121a3400
2019-04-05 01:42:22.024 -0600 connmgr: connection entry not removed for pending refcount. devid=log-collector cesock=50 sockfd=4294967295 refcount=1
2019-04-05 01:42:22.024 -0600 Error: pan_cfg_log_buffer_nsend(pan_cfg_log_buffer.c:172): Failed to send the logrec to log collector
2019-04-05 01:42:22.024 -0600 Error: pan_log_buffer_cursor_next(pan_cfg_log_buffer.c:1538): logbuffer: failed to send 10 'traffic' logs to cms
2019-04-05 01:42:22.024 -0600 COMMS: ssl write - shutdown exit. sock=50 err=0 sslerr=1 errnum=336396495(protocol is shutdown)
2019-04-05 01:42:22.024 -0600 Error: cs_msg_tcp_send_ex(cs_transport.c:154): COMM: failed to send header. result=0 len=12 ctype=3 dtype=7 mtype=0 sock=50 ssl=0x121a3400
2019-04-05 01:42:22.024 -0600 Error: pan_lcsa_tcp_channel_loop(src_panos/lcs_agent.c:2678): lcs agent: failed send probe. tcp send failure. sock=50 ssl=0x121a3400
2019-04-05 01:42:22.024 -0600 connmgr: connection entry removed. devid=log-collector sock=50 result=0
2019-04-05 01:42:22.024 -0600 connmgr: unlock - remove conn entry. devid=log-collector sock=50 result=0
2019-04-05 01:42:22.024 -0600 lcs agent: peer watch. sock=50 curtime=25052064 recvtime=25051889 proctime=25051889 sendtime=25052064 errcount=1
2019-04-05 01:42:22.024 -0600 COMMS: ssl read - zero byte. sock=50 err=0 sslerr=6 errnum=0((null))
2019-04-05 01:42:22.024 -0600 Error: cs_recv_tcp_data(cs_comm_utils.c:641): COMMS: could not read header. sock=50 ssl=0x121a3400 len=-1 hdrlen=0

MP

Help the community: Like helpful comments and mark solutions.
1 accepted solution

Accepted Solutions

@MP18,

Best practice on this would to actually seperate out interfaces for log collection, and a seperate one for collector group comms. Depending on how many devices you have communicating to the M500 you actually might benefit from having multiple interfaces configured from log collection, not just a single interface. 

 

Documentation on the Log Collector interface settings can be found starting HERE in the documentation. 

View solution in original post

6 REPLIES 6

Cyber Elite
Cyber Elite

@MP18,

Based off the logs it looks like the connection to 10.7.1.139 is spotty and the firewall can't form a good connection to the log collector. Is this working now or has it stayed disconnected? 

now all the firewalls are connected to log collector.

this happended few times tonight.

 

is this physical connection issue or some bug?

MP

Help the community: Like helpful comments and mark solutions.

i checked the network each device and interface between PA and log collectores no issues.

MP

Help the community: Like helpful comments and mark solutions.

Seems we are only using management interface of m500.

How can i use other interfaces of m500 to collect logs from the firewalls?

 

 

MP

Help the community: Like helpful comments and mark solutions.

@MP18,

Best practice on this would to actually seperate out interfaces for log collection, and a seperate one for collector group comms. Depending on how many devices you have communicating to the M500 you actually might benefit from having multiple interfaces configured from log collection, not just a single interface. 

 

Documentation on the Log Collector interface settings can be found starting HERE in the documentation. 

Will do this now going forward.

Thanks for confirming that.

MP

Help the community: Like helpful comments and mark solutions.
  • 1 accepted solution
  • 10744 Views
  • 6 replies
  • 0 Likes
Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!

The LIVEcommunity thanks you for your participation!