Skip to main content.
home | support | download

Back to List Archive

Re: FileRules

From: Michael <michael(at)not-real.bizsystems.com>
Date: Wed Jun 16 1999 - 22:42:53 GMT
> I believe someone posted a patch recently that did exactly this.
> 
Nope, that patch is for the same problem, but it is in 
http.c and I've applied it.

The problem is that the perl helper 'swishspider' fails the test on 
the "text/html" string and NEVER PASSES the page to swish-e for 
parsing. The result being that all the links on the page are skipped.

I have not tried it with the patch to http.c removed, but looking at 
the code, it seems to me it would be a second point of failure.

> At 01:19 PM 6/15/99 -0700, Michael wrote:
> >I'd like to be able to use the FileRules when indexing http sites. Is 
> >there a simple method to enable these checks for spidering??
> >
Michael@bizsystems.com
Received on Wed Jun 16 14:40:00 1999