Skip to main content.
home | support | download

Back to List Archive

Re: Re: Any way to restrict URLs with http method

From: <john.leth(at)>
Date: Fri Apr 30 1999 - 13:15:13 GMT
I did overstate the second item. These sites are behind a firewall --
Apache is set to be fairly secure in what it sends in the first place but
content providers are newbies for the most part. I prefer to err on the
side of paranioa.

Ron Klatchko wrote:

> At 05:34 PM 4/29/99 -0400, wrote:
> >o Stop indexing of anything that might possibly be under construction,
> >confidential, or an outright security risk.
> Wouldn't you want those excluded by robots.txt anyway?  Otherwise
> commerical search engines will index them.
> moo
> ----------------------------------------------------------------------
>           Ron Klatchko - Manager, Advanced Technology Group
>            UCSF Library and Center for Knowledge Management
Received on Fri Apr 30 06:11:19 1999