> I believe someone posted a patch recently that did exactly this.
Nope, that patch is for the same problem, but it is in
http.c and I've applied it.
The problem is that the perl helper 'swishspider' fails the test on
the "text/html" string and NEVER PASSES the page to swish-e for
parsing. The result being that all the links on the page are skipped.
I have not tried it with the patch to http.c removed, but looking at
the code, it seems to me it would be a second point of failure.
> At 01:19 PM 6/15/99 -0700, Michael wrote:
> >I'd like to be able to use the FileRules when indexing http sites. Is
> >there a simple method to enable these checks for spidering??
Received on Wed Jun 16 14:40:00 1999