Navigation
3 users online :: 3 Guests and 0 Registered
Records in this category
Tags
Sticky FAQs
Robots.txt
ID #1017
A Blank Robots.txt File
If you want to allow robots/spiders to index all of the content it finds on your webpage (i.e. all linked content or ocntent listed on your sitemap) then you can use the following in your robots.txt file .
User-agent: *
Disallow:
It is prefered for Google that you have either a blank robots.txt file, or you have the content above. If the request for the robots.txt file on your domain returns an error or a redirect status, your domain is at risk of being removed from the index. Take our advice and at least use the above.
Tags: robots.txt, seo
Related entries:
Last update: 2008-08-18 22:09
Author: Paul Savage
Revision: 1.0
You can comment this FAQ