Skip to main content.
home | support | download

Back to List Archive

Re: RE: LWP,HTTP and HTML modules

From: Yann Stettler <stettler(at)>
Date: Tue Jan 19 1999 - 21:12:33 GMT
Ron Klatchko wrote:
> I find this a slightly curious statement.  If you control the extensions,
> then I assume you control the web site.  If what you are looking for is to
> make the indexing more efficient, wouldn't it make more sense to use the
> file system methods which will always blow away the HTTP method is indexing
> speed?

No because the filesystem method as the following problem :
1) It will index files that aren't linked anymore. So people will
   be able to access out-of-date documents, draft, junk left
   in directory and such

2) Many site used database to store large amount of document. You
   can't index them by the filesystem. You need to use the HTML

3) Some part of the website may be password protected. Using the
   filesytem will allow access to the files (or at least show that
   those documents exist).

Yes, the filesystem method will always be faster than the HTTP
one... but with a good server, going through the loopback
interface, it won't even be that significatif especialy if you
discard (as the filesystem method do), files that you know shouldn't
be indexed anyway.

Yann Stettler

TheNet - Internet Services AG              CohProg SaRL                           
Anime and Manga Services         
Received on Tue Jan 19 13:05:22 1999