Skip to main content.
home | support | download

Back to List Archive

Re: Any way to restrict URLs with http method since you can't use FileRules?

From: Ron Klatchko <ron(at)not-real.library.ucsf.edu>
Date: Thu Apr 29 1999 - 21:06:26 GMT
My original design assumed you would control that with robots.txt.  You can
use the useragent name of swishspider.

moo

At 01:49 PM 4/29/99 -0700, john.leth@gulfaero.com wrote:
>Any way to restrict URLs when using http method since you can't use
>FileRules Directive?
>
>I'd like to be able to stop indexing of certian areas. My guess is that
>I will have to modify
>the swishspider perl program. Has anyone else done this already.
>
>-John Leth-Nissen
>Gulfstream Aerospace Corp.
>
>
>
>
----------------------------------------------------------------------
          Ron Klatchko - Manager, Advanced Technology Group           
           UCSF Library and Center for Knowledge Management           
                        ron@library.ucsf.edu                
Received on Thu Apr 29 14:03:01 1999