Google Product Forums

How long to recrawl / Made changes and not crawled yet / Request Crawling / Request Recrawl ::: Auto-Response :::


Autocrat 2010/06/02 8:41
グループへの投稿: Webmaster Central

カテゴリ: Crawling, indexing & ranking :

Please - do Not post your questions in this topic!
------------------------------------------------------------
This is an Auto-Response for the (fairly) common question/issue regarding getting recrawled.
This is an attempt to compile information for questions/issues such as;

How long till recrawled
How long till the next crawl
How to request a recrawl
How to get recrawled sooner/faster
Made changes - how do I get it recrawled
Made changes - why is it not updated yet
etc. etc. etc.


=============   It Varies!   =============

There is no specific, hard set, predefined "time" between crawls that applies to all sites.
It varies, from site to site.
It varies from page to page.
It is based on Popularity.
(The publicly visible (and often inaccurate) measurement is PR)

If your page/site is Not popular - then it will take longer till it is crawled again.


=============   How to Ask for it?   =============

You don't.
It is Automated.

As per above - Google schedules it based on Popularity.
You do Not fill in a form and ask them to crawl page X on site Y.

You wait for it to happen.


=============   Why can't I ask for a recrawl?   =============

You're having a laugh - right?
Can you imagine how much/how often that would be used?
Google would hardly have time to crawl new stuff,
simply because people fixed a spelling mistake, bad link or people removing spam off their pages!

Do as suggested above - then you wait!


=============   Nothing I can do?   =============

Well - I didn't actually say that!

I said it's automated based on Popularity.
(Yes - I'm repeating my self ... that's because people seem to have a hard time taking hints/absorbing what they are told!)

What you may beable to do is "encourage" it to happen sooner/faster.

We know it's based on popularity ... so we are looking at making it more popular.

First things first - make sure you have pages on your site linking to it.
If the page only has a few internal links - then it isn't likely to be popular/important.
So - put links to that page from the Homepage and any other high-traffic/important/popular page.

After that - see about an External Link or two to it.
That may help increase the PR (not the one you see - that takes time to update),
but G has it's own records that update more often,
and should help.

Additionally - you should ensure that your server/script uses the LastModified header() value,
and that it is correct.
That way - if G decides to pay attention and see if the page has changed,
it may entice it to come look.

You can also try using a Sitemap.
Though they are optional - and G doesn't have to crawl the listed URLs,
it "may" (possibly, at a rare chance!) help.
You should also consider using hte LastMod sitemap value.
(As far as we know - G doesn't bother using it as most folk get it wrong/blatantly lie with it),
but it's worth a shot.


=============   !!! But it's URGENT !!!   =============

Then I suggest you go look at hte FAQs for Removing Content!
There is the URL Removal Request tool.
It has an option for stuff that has Changed.

That means things like the Description/Cache may be removed,
and pend an update.

Make sure you READ The Instructiosn for the URL Removal Request tool!
And if you want help with that - post in the Removal Group!


---------------------------------------------------------------------


NOTE:

This is a "general auto-response" post.

This is Not a Topic for discussion;

It is a point of reference to save having to type the same answer repeatedly due to the sheer number of times this question is asked and is meant as an aid for people that don't seem search/read the various other posts regarding this topic.
Thank you for taking the time to read this Auto-Response.

Please - do Not post your questions in this topic!