Skip to main content.
home | support | download

Back to List Archive

Trouble with obeyRobotsNoIndex

From: Markus Strickler <markus(at)not-real.braindump.ms>
Date: Wed Oct 16 2002 - 16:26:58 GMT
Hi-

I'm currently encountering problems with swish-e not generating an index
if I have obeyRobotsNoIndex set to yes and use the HTML2 parser.
With swish-e-2.1-dev-25-2002-03-22 which worked for several months
without any problems indexing just stops during writing word data.
So I tried the latest release for windows (2.2.1) which stops during
indexing after a few hundred files with "Indexing failed".
The directory I try to index contains around 700 files where most
contain a robots no index meta tag.
Has anybody else had similar problems? I've tried this on three machines
all running windows 2000 without success.

-markus  


 
Received on Wed Oct 16 16:30:58 2002