Dear Palo Alto Networks,
Off late, I have been seeing some bizzare issues with Brightcloud services version 3954 and 3955. As per what I could see under the system logs, on 09/10/12, the brightcloud server could not receive data and there was failure in retrieving data (in-spite of having all the network connections checked and were absolutely fine). Then on 10/10/12, connection fails to Brightcloud server causing major issues across various locations where PAN has been deployed.
The issue was many educational institues block the category "not-resolved" and with Brightcloud going down and with no Dynamic dB query (in-spite of having Dynamic URL Filtering under URL Filtering profile checked) did cause havoc.
Please see snapshots below:
Can someone please let me know why all this has happened? Was any kind of maintenance work taking place? If yes, did I miss any warnings from PAN (as per my knowledge, I don't think so).
I know version 3956 is out and the first thing I will be told is to upgrade which I will be doing it or rather done it as I am writing this.
But I would really appreciate if someone kindly explains the reason behind this issue. :smileyhappy:
Look forward to hearing from PAN.
There was indeed an outage at BrightCloud yesterday - they're currently working with their service providers to identify the root cause of the failure and working on improving their failover procedures to reduce the impact on customers such as yourself. I'm hopeful that the chances of this type of outage will be minimized in the future and appreciate everyone's patience and understanding as BrightCloud works to resolve their issues.
admin@PAN-Primary(active)> tail follow yes mp-log pan_bc_download.log
Oct 11 14:02:25 Cannot receive data from 'service.brightcloud.com:80' to download BrightCloud URL database
Oct 11 14:02:25 Error downloading latest URL database
Oct 16 07:29:10 ip 126.96.36.199 message RT time 0.046
Oct 16 07:29:10 ip 188.8.131.52 message RT time 0.013
Oct 16 07:29:10 ip 184.108.40.206 message RT time 0.019
Oct 16 07:29:10 ip 220.127.116.11 message RT time 0.020
Oct 16 07:29:10 Best IP for service.brightcloud.com is 18.104.22.168
Oct 16 07:29:10 Connected to Brightcloud update server service.brightcloud.com
Oct 16 07:29:10 Cannot receive data from 'service.brightcloud.com:80' to download BrightCloud URL database
Oct 16 07:29:10 Error downloading latest URL database
We also have problems with Brightcloud Updates.
Are there any know issues at the moment?
Oct 17 16:57:47 ip 22.214.171.124 message RT time 0.336
Oct 17 16:57:48 ip 126.96.36.199 message RT time 0.329
Oct 17 16:57:48 ip 188.8.131.52 message RT time 0.061
Oct 17 16:57:48 ip 184.108.40.206 message RT time 0.350
Oct 17 16:57:48 Best IP for service.brightcloud.com is 220.127.116.11
Oct 17 16:57:48 Connected to Brightcloud update server service.brightcloud.com, initiated by 172.25.102.2
Oct 17 16:57:49 Newer update available...
Oct 17 16:57:49 ip 18.104.22.168 message RT time 0.531
Oct 17 16:57:50 ip 22.214.171.124 message RT time 1.114
Oct 17 16:57:51 ip 126.96.36.199 message RT time 0.230
Oct 17 16:57:51 ip 188.8.131.52 message RT time 0.163
Oct 17 16:57:51 Best IP for database.brightcloud.com is 184.108.40.206
Oct 17 16:57:51 Connected to Brightcloud update server database.brightcloud.com, initiated by 172.25.102.2
Oct 17 16:57:51 Error: Missing Content-Length:
HTTP/1.1 200 OK
Date: Wed, 17 Oct 2012 14:57:50 GMT
Oct 17 16:57:51 Failed to download 'part_bcdb_3.961.bin'
Oct 17 16:57:51 Error: pan_bc_download(pan_bc_url.c:1506): Failed to perform download and update
Oct 17 16:57:51 Error downloading latest URL database
We recently purchased the URL Filtering package, but have been running a PAN for the last 2 years.
To get it to work I had to set the Service Route Configuration in setup to connect with my public interface.
Well, it will be more useful if the cause behind the outage was disclosed as I have people on my back asking for the reason. :smileyhappy:
With regards to error in downloading the URL database, this seems to a mixed bag now..
I am gettign the same error when trying to downlaod a new URL signature. We are running 4.1.6 and have not been able to update from 3995.:
BrightCloud URL filtering database update initiated
(active)> tail follow yes mp-log pan_bc_download.log
Jan 22 10:54:13 Newer update available...
Jan 22 10:54:15 ip 220.127.116.11 message RT time 0.377
Jan 22 10:54:15 ip 18.104.22.168 message RT time 0.083
Jan 22 10:54:16 ip 22.214.171.124 message RT time 0.444
Jan 22 10:54:17 ip 126.96.36.199 message RT time 1.166
Jan 22 10:54:18 ip 188.8.131.52 message RT time 0.815
Jan 22 10:54:18 Best IP for database.brightcloud.com is 184.108.40.206
Jan 22 10:54:18 Connected to Brightcloud update server database.brightcloud.com, initiated by 192.168.10.14
Jan 22 10:54:18 URL database download: 0% done
Jan 22 10:54:22 URL database download: 1% done
Jan 22 10:54:25 URL database download: 2% done
Jan 22 10:54:29 URL database download: 3% done
Jan 22 10:54:33 URL database download: 4% done
Jan 22 10:54:36 URL database download: 5% done
Jan 22 10:54:39 URL database download: 6% done
Jan 22 10:54:43 URL database download: 7% done
Jan 22 10:54:46 URL database download: 8% done
Jan 22 10:54:49 URL database download: 9% done
Jan 22 10:54:53 URL database download: 10% done
Jan 22 10:54:56 URL database download: 11% done
Jan 22 10:54:59 URL database download: 12% done
Jan 22 10:55:03 URL database download: 13% done
Jan 22 10:55:07 URL database download: 14% done
Jan 22 10:55:10 URL database download: 15% done
Jan 22 10:55:13 URL database download: 16% done
Jan 22 10:55:16 URL database download: 17% done
Jan 22 10:55:20 URL database download: 18% done
Jan 22 10:58:16 Failed to download 'full_bcdb_4.28.bin'
Jan 22 10:58:16 Error: pan_bc_download(pan_bc_url.c:1501): Failed to perform download and update
Jan 22 10:58:16 Error downloading latest URL database
Received header HTTP/1.1 200 OK
Date: Tue, 22 Jan 2013 15:53:08 GMT
error in SSL_read() call
Click Accept as Solution to acknowledge that the answer to your question has been provided.
The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!
These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the Live Community as a whole!
The Live Community thanks you for your participation!