# See http://web.nexor.co.uk/mak/doc/robots/norobots.html for a description # of what this file is, and it's syntax. - CAD 4/30/95 # # Generally, it's: # # User-agent: agent-name # Disallow: directory on the server you wish to not have robots in # # To keep all robot's out of directory "blah", use: # # User-agent: * # Disallow: /blah/ # # CAD 4/30/95 I think that robots should stay our of the cgi-bin directory: User-agent: * Disallow: /orig_content/ Disallow: /_root/