Robots.txt

ID #1017

A Blank Robots.txt File

If you want to allow robots/spiders to index all of the content it finds on your webpage (i.e. all linked content or ocntent listed on your sitemap) then you can use the following in your robots.txt file .

 User-agent: *

 Disallow:

It is prefered for Google that you have either a blank robots.txt file, or you have the content above. If the request for the robots.txt file on your domain returns an error or a redirect status, your domain is at risk of being removed from the index. Take our advice and at least use the                 above.

Tags: seo, robots.txt

Related entries:

Last update: 2008-08-18 22:09
Author: Paul Savage
Revision: 1.0

Digg it! Print this record Send to a friend Show this as PDF file
Propose a translation for Propose a translation for
Please rate this entry:

Average rating: 0 out of 5 (0 Votes )

completely useless 1 2 3 4 5 most valuable

You can comment on this entry