nice SEO FAQ

ID #1018

It may happen that you have certain sub directories that you don't want to be listed in the search engines. All major search engines that follow the REP , i.e. Yahoo, MSN and Google, and the promise not to index folders and files that are specifically mentioned on your robots.txt file. Here is an example to excluse subfolders /private/ /photos-saved/ and /images-fullsize/ from the search index,

User-agent: *
Disallow: /private/
Disallow: /photos-saved/
Disallow: /images-fullsize/

By using User-agent: * it referes to the fact that all search engines should obey this request.  

Please rate this FAQ:

Average rating: 0 (0 Votes)

completely useless 1 2 3 4 5 most valuable

comment this FAQ

Your Name:
Your email address:
Your comment:

Please enter the characters you're reading in the image