Skip to main content.
home | support | download

Back to List Archive

Re: Duplicate Documents and Digest::MD5

From: Bill Moseley <moseley(at)not-real.hank.org>
Date: Fri Oct 01 2004 - 14:38:59 GMT
On Fri, Oct 01, 2004 at 07:35:18AM -0700, Jon Sorensen wrote:
> Duplicate Documents
> 
> in http://swish-e.org/current/docs/spider.html says to use
> 
> Digest::MD5 to fingerprint indexed pages and prevent indexing duplicates
> pages
> 
> Do I *just* need to install the module for this to work? I'm not  clear on
> this

^F md5 in the browser

there's a use_md5 option.

-- 
Bill Moseley
moseley@hank.org

Unsubscribe from or help with the swish-e list: 
   http://swish-e.org/Discussion/

Help with Swish-e:
   http://swish-e.org/current/docs
   swish-e@sunsite.berkeley.edu
Received on Fri Oct 1 07:39:08 2004