Categories: Crawling, indexing & ranking :

unwanted CMS pages are being indexed

Showing 1-2 of 2 messages
unwanted CMS pages are being indexed k3nc 3/19/13 8:56 AM
I've read the FAQs and searched the help center. 


google is indexing a bunch of pages which are the by product of product filtering page, which have the format www.mywebsite/explore/index/loaddata/id/10/. 

These pages have no content other than images - also the pages don't have page titles and are therefore being flagged in webmaster tools.

Should I exclude them via robots.txt or take another approach, like not worry about them :-)

many thanks

Re: unwanted CMS pages are being indexed bperrotin 3/19/13 9:45 AM
Hi k3nc, 

Seems those pages are not relevant :) 

Maybe your could try by using HTTP headers within your .htaccess: 

RewriteCond %{REQUEST_URI} ^/explore/index/loaddata/id/.*$ 
RewriteRule . - [E=headernoindex] 
Header set X-Robots-Tag "noindex" env=headernoindex

Double security: block them in the robots.txt:
Disallow: /explore/index/loaddata/id/