Skip to main content.
home | support | download

Back to List Archive

Re: test & rewrite?

From: <dhane(at)not-real.pw.usda.gov>
Date: Sat Nov 12 2005 - 16:18:17 GMT
On Friday 11 November 2005 12:35, François Tissandier wrote:
> Hello
>
> I have a tricky problem I can't solve:
>
> I have pages like this:
> www.blabla.com/tutu/index.html
> www.blabla.com/index.html
>
> Basically, it's the same page, except for some content that I don't want
> to index. So I want those two pages to be indexed as one.
>
> Sadly, the content is slightly different, so the MD5 function is not
> working. So I was wondering if it would be possible to give some rules to
> the spider to tell him
>
> "link "/tutu/index.html" is OK, but follow "/index.html" instead, please,
> will you?"
>
> Thanks in advance for your help!
>
>       François


Couldn't you just use your robots.txt file to tell the spider to skip over the 
one file?

Dave
Received on Sat Nov 12 08:18:27 2005