Skip to main content.
home | support | download

Back to List Archive

tracking the completeness of spidering?

From: Richard Morin <rdm(at)not-real.slac.stanford.edu>
Date: Thu Dec 16 2004 - 19:25:28 GMT
Some pages seem to be missing from my indexes, so I'm
wondering if the spidering process is failing in some
silent manner.

I have considered adding some code to spider.config
(e.g., to tally visited pages and report the results),
but before I go there, I thought I should ask if there
is an easier way.  Clues?  Suggestions?

-r
Received on Thu Dec 16 11:25:34 2004