Whitelisting socketTimeout parameter and added Googlebot

Open kartikl opened this issue 8 years ago • 2 comments

Nov 08 '17 18:11 kartikl

You should not add the Googlebot user agent to the list of user agents being checked. Checking Googlebot by their user agent could get you penalized for cloaking.

Google follows the escaped fragment crawling protocol so if you add <meta name="fragment" content="!"> to the <head> of your pages then Google will access the ?_escaped_fragment_= URL

Nov 08 '17 20:11 thoop

@thoop thanks. Will remove googlebot

Nov 10 '17 17:11 kartikl