Categories: Sitemaps :

Network Unreachable/Robots.txt

Showing 1-8 of 8 messages
Network Unreachable/Robots.txt Biggy Fat 9/13/09 1:39 PM
Hello World.  I am having a problem with trying to get Google to read my sitemap.  I have submitted and I have come up with this error:

Network unreachable: robots.txt unreachable
We were unable to crawl your Sitemap because we found a robots.txt file at the root of your site but were unable to download it. Please ensure that it is accessible or remove it completely.

There wasn't even a robots.txt on my site at the time but I put one on using the tool that Google provides and uploaded it at the root but I STILL got the same error.  Please help!
Re: Network Unreachable/Robots.txt webado 9/13/09 1:58 PM
When the server is unreachable the robtos does nto know whether there is a robtos.txt file or not - all it knows is that it cannot reach it or any part of the server.
 
This means your server/firewall are blocking Googlebot. Or else your domain has a bad DNS zone.
 
Cannot offer more help without knowing your website url.
Re: Network Unreachable/Robots.txt Biggy Fat 9/13/09 2:11 PM
URL is http://www.bestdesktopcomputer.org.
Re: Network Unreachable/Robots.txt webado 9/13/09 2:39 PM
Your server responds with a 500 Internal server error when you try to access the robots.txt file:
 
 
 
If the file does not exist it should  respond with a 404 Not found.
Re: Network Unreachable/Robots.txt Biggy Fat 9/13/09 4:04 PM
That's the thing, though - even when I DO have the robots.txt file in the server I still get the same error.
Re: Network Unreachable/Robots.txt webado 9/13/09 7:03 PM
So your server is badly  scewed up. Talk to your hoster.
 
Maybe the site was hacked. You have to check the whole hosting account using FTP and see what's up with everythign starting with any .htaccess files in the root and any folders .
Re: Network Unreachable/Robots.txt Biggy Fat 9/13/09 7:39 PM
Well, it seems to be fine now, though it was fine two days ago and all of a sudden it went bad again.  I'll ask again if it happens.
Re: Network Unreachable/Robots.txt webado 9/13/09 7:54 PM
But your server is still misconfigured and responds with a 500 for any non-existent file:
 
That should be a 404.
 
If you don't get thta fixed you will have no end of problems.