topic Hosting Websites - Google &quot;Crawling&quot; issues... in General Topics https://live.paloaltonetworks.com/t5/general-topics/hosting-websites-google-quot-crawling-quot-issues/m-p/45330#M33316 <HTML><HEAD></HEAD><BODY><P>I haven't found the cause just yet, but I've noticed that since implementing our PAN firewalls, (PA-2020) I've noticed in our Google Webmaster tools that Google has stopped indexing our websites.&nbsp; This is causing me big grief as you could imagine. Now, I am allowing incoming "web-browsing" but I have to wonder if somehow the firewall is blocking robots...&nbsp; I've turned on incoming logging to see if I can figure this out.&nbsp; Any tips would be appreciated.</P></BODY></HTML> Mon, 15 Aug 2011 22:50:22 GMT cenders 2011-08-15T22:50:22Z Hosting Websites - Google "Crawling" issues... https://live.paloaltonetworks.com/t5/general-topics/hosting-websites-google-quot-crawling-quot-issues/m-p/45330#M33316 <HTML><HEAD></HEAD><BODY><P>I haven't found the cause just yet, but I've noticed that since implementing our PAN firewalls, (PA-2020) I've noticed in our Google Webmaster tools that Google has stopped indexing our websites.&nbsp; This is causing me big grief as you could imagine. Now, I am allowing incoming "web-browsing" but I have to wonder if somehow the firewall is blocking robots...&nbsp; I've turned on incoming logging to see if I can figure this out.&nbsp; Any tips would be appreciated.</P></BODY></HTML> Mon, 15 Aug 2011 22:50:22 GMT https://live.paloaltonetworks.com/t5/general-topics/hosting-websites-google-quot-crawling-quot-issues/m-p/45330#M33316 cenders 2011-08-15T22:50:22Z Re: Hosting Websites - Google "Crawling" issues... https://live.paloaltonetworks.com/t5/general-topics/hosting-websites-google-quot-crawling-quot-issues/m-p/45331#M33317 <HTML><HEAD></HEAD><BODY><P>We had another customer with a similar issue.&nbsp; He had allowed web-browsing, ssl, flash aand ping.&nbsp; Once he added the application web-crawler to the policy, the Google fetches (request for index) were successful.&nbsp; Hope this helps.</P></BODY></HTML> Mon, 15 Aug 2011 23:50:26 GMT https://live.paloaltonetworks.com/t5/general-topics/hosting-websites-google-quot-crawling-quot-issues/m-p/45331#M33317 nrice 2011-08-15T23:50:26Z Re: Hosting Websites - Google "Crawling" issues... https://live.paloaltonetworks.com/t5/general-topics/hosting-websites-google-quot-crawling-quot-issues/m-p/45332#M33318 <HTML><HEAD></HEAD><BODY><P>Awesome. Thank you. Had just started seeing that in the logs (I normally don't log blocked traffic coming into the dmz).</P></BODY></HTML> Tue, 16 Aug 2011 01:55:55 GMT https://live.paloaltonetworks.com/t5/general-topics/hosting-websites-google-quot-crawling-quot-issues/m-p/45332#M33318 cenders 2011-08-16T01:55:55Z