Skip to main content.
home | support | download

Back to List Archive

Duplicate files

From: GUEGAN Ronald <rguegan(at)>
Date: Fri Apr 26 2002 - 16:05:29 GMT

Is there a way to detect that an HTML file as already been indexed ?

We are indexing websites where a file can be accessed in various way :
In the given example, both url could point to the same page.

Can we specify any criteria (page size, title, description, ) so the spider
can detect it's the same page ?

   Thanks in advance.

Received on Fri Apr 26 16:05:46 2002