Skip to main content.
home | support | download

Back to List Archive

Re: Does the <!-- Swishcommand noindex --> work whe

From: Bill Moseley <moseley(at)>
Date: Wed Jun 25 2003 - 15:53:04 GMT
On Wed, Jun 25, 2003 at 10:15:10AM -0500, wrote:
> > /var/www is the document root.
> Is robots.txt just a linux thing? I've got swish-e running in redhat 8,
> and it's spidering a site on a win2k box.

I assume you mean you are running IIS on the Windows machine?  Do you 
see the request for robots.txt in the IIS access log? 


moseley(at)not-real.bumby:~/apache$ GET
User-agent: *

Disallow: /citydirs/1857/1857full.pdfmoseley@bumby:~/apache$

Maybe you need a line feed after that line?

Bill Moseley
Received on Wed Jun 25 15:53:13 2003