Google Product Forums

Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content


Richard Hearne Mar 15, 2011 3:17 AM
Posted in group: Webmaster Central

Categories: Crawling, indexing & ranking :

>> There are loads of pages still in the index that have been added to my robots.txt disallow list for days. 

I haven't followed this thread completely, but I don't think you should use robots.txt to remove any content.  It's doesn't remove content, it simply blocks access.  If you add Robots META NoIndex, and block access nothing will happen.

I'd strongly suggest you remove bad content ASAP, and this means NOT using robots.txt for this.