View Full Version : create a robots.txt file
03-08-2010, 12:15 PM
I keep getting 404 error code on my web site. 95% of the codes is due to robots.txt not being found on my site. Trying to find out how to edit or make a robots.txt file and where to place within my web site. Any help would be greatly appricated.
03-08-2010, 01:10 PM
the robots.txt should be in the root folder i.e. residing alongside your home page.
Here is a link that tells you all about the robots.txt file and what you can or can't do and how to.
and here is a google link which also explains that if you do exclude pages etc they still maybe found ! i.e. Dont rely on the robots.txt to exclude files, becuase there is no guarantee.
Hope this helps
Powered by vBulletin® Version 4.1.12 Copyright © 2013 vBulletin Solutions, Inc. All rights reserved.