On Wed, Jun 25, 2003 at 10:15:10AM -0500, Cleveland@mail.winnefox.org wrote:
> > /var/www is the document root.
> Is robots.txt just a linux thing? I've got swish-e running in redhat 8,
> and it's spidering a site on a win2k box.
I assume you mean you are running IIS on the Windows machine? Do you
see the request for robots.txt in the IIS access log?
moseley(at)not-real.bumby:~/apache$ GET http://www.oshkoshpubliclibrary.org/robots.txt
Maybe you need a line feed after that line?
Received on Wed Jun 25 15:53:13 2003