Skip to main content.
home | support | download

Back to List Archive

Duplicate files

From: GUEGAN Ronald <rguegan(at)not-real.sigma.fr>
Date: Fri Apr 26 2002 - 16:05:29 GMT
 Hi,

Is there a way to detect that an HTML file as already been indexed ?

We are indexing websites where a file can be accessed in various way :
  - http://www.mysite.com/app1/page.asp?param=1&other=0
  - http://www.mysite.com/app1/page.asp?param=1
In the given example, both url could point to the same page.

Can we specify any criteria (page size, title, description, ) so the spider
can detect it's the same page ?

   Thanks in advance.

           Ronald.
Received on Fri Apr 26 16:05:46 2002