Skip to main content.
home | support | download

Back to List Archive

Re: Does the <!-- Swishcommand noindex --> work whe

From: <Cleveland(at)>
Date: Wed Jun 25 2003 - 17:57:29 GMT
> Ah, yes.  remove your blank lines.  A blank line separates sections 
> based on the user agent.  

Hm. Still not working. I looked at cnn's robots.txt and I noticed they
didn't have multiple directories listed. Just /directory or file.html,
not /directory/directory/file.html. Is it ok to put sub folders? Also, I
have the spider only looking at
Could that be the problem?

Jody Cleveland
Received on Wed Jun 25 17:57:35 2003