Google Product Forums

Re: Think you're affected by the recent algorithm change? Post here.

Robert - NoVa Mar 15, 2011 1:25 PM
Posted in group: Webmaster Central

Categories: Crawling, indexing & ranking :

I don't know if this is related, but .. starting Feb 27, Googlebot began finding thousands of bogus urls on my site, such as

http://mc-computing.com/Databases/MySQL/Parasites/Science_Facts/Lapse_Rate/blogs/ISPs/WordPress/qs/Global_Warming/NewsPapers/Lohachara

However, this is a composite of several valid urls.

http://mc-computing.com/Databases/MySQL
http://mc-computing.com/Science_Facts/Lapse_Rate
http://mc-computing.com/blogs
http://mc-computing.com/ISPs/WordPress
http://mc-computing.com/qs/Global_Warming/NewsPapers/Lohachara.html

This was not a problem previously. I also got the following

  "Restricted by robots.txt ‎(24,468)"

The value should be under 300. Apparently, since google now thinks that my site is some kind of spammer (or something), my search ranks have tanked and my alexa ranging has tanked.

This appears to also be associated with my site no longer being "verified". (I have fixed that, but a explanation on the site would have been nice.)

Also, once this is fixed, how do I clear the robots cache so that only valid data is displayed?