03-05-2014 09:48 AM
I found reference to this discussion but there were no details specifying how to allow googlebot crawls only.
How to Allow Googlebot Through the Firewall
I would like to only allow google and bing if that's possible. I can see options for google-analytics in the definition rules. Can this be done?
03-05-2014 01:57 PM
I don't think if there is a specific application for googlebot crawlers. Will let someone answer this.
I am sure this is not an optimal solution, But did you consider using URL filtering, or Destination IP address/FQDN object?
04-11-2014 12:00 PM
Sorry for the late reply prb. I decided for now to enable crawling. I thought perhaps there was more granular way to do it.
02-25-2015 08:49 AM
This discussion is the closest that we can do at this time:
Re: web crawling for google only
Past this, would be limiting it by IP only for application Web-Crawler.
The request for a reverse lookup or a way to control the Spider/web crawlers would be a feature request that you need to talk with your SE about.
I hope this helps.
-JoeD
Click Accept as Solution to acknowledge that the answer to your question has been provided.
The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!
These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!
The LIVEcommunity thanks you for your participation!
