Google Product Forums

Re: Why will google not crawl anything that's in my movable type directory structure?

unclehighbrow Jan 10, 2010 10:10 AM
Posted in group: Webmaster Central

Categories: Crawling, indexing & ranking :

Right, diogenes club does show up in searches, but only the homepage and a few html files I've added myself. Nothing from the movable type date or category archives ever shows up.  So basically, once a post leaves the home page, it disappears from google forever.  Some examples:

http://www.diogenesclub.net/archives/2008/05/my_mustache_fin.html
http://www.google.com/search?client=safari&rls=en&q=%22My+mustache+finally+came+in+handy%22&ie=UTF-8&oe=UTF-8

http://www.diogenesclub.net/archives/2006/02/i_went_to_sunda.html
http://www.google.com/search?client=safari&rls=en&q=%22I+went+to+Sundance+and+found+God+%22&ie=UTF-8&oe=UTF-8

http://www.diogenesclub.net/archives/2008/12/pay_attention_c.html
http://www.google.com/search?client=safari&rls=en&q=%22Pay+attention,+Congress+and+auto+industry,%22&ie=UTF-8&oe=UTF-8