Google Product Forums

Re: Huge Spike in Traffic from Direct Internet Explorer visitors - nearly100% bounce rate, stays for only one minute


PPC_Guru Feb 27, 2012 4:04 PM
Posted in group: Google Analytics

Categories: Discuss Tracking and Implementation issues :

This is a bit of a long shot... but here is a robots.txt whitelist: http://db.tt/wd4WUCKp

If this is server monitoring service (rather than malware), then there is a chance that it might obey the User-agent: * Disallow: /

Robots.txt file to upload to root:
http://db.tt/wd4WUCKp

Note1: change yourdomain within Sitemap: http://www.yourdomain.com/sitemap.xml
Note2: www.facebook.com/robots.txt also uses a robots.txt whitelist.

Thanks

Phil.

Also, here are more examples of robot spider-traps...
http://www.webproworld.com/webmaster-forum/threads/58822-Prevent-referrer-spam-block-bad-bots-from-stealing-bandwith
http://danzcontrib2.free.fr/en/pieges.php#bloquer