Skip to main content.
home | support | download

Back to List Archive

Re: RE: LWP,HTTP and HTML modules

From: Yann Stettler <stettler(at)>
Date: Tue Jan 19 1999 - 21:06:59 GMT
Ron Klatchko wrote:

> I don't see this as strengthening your suggestion.  If a URL doesn't have

I don't think that I need to strengthen anything. I suggest
the possibility of using a configuration option in some case.
If you don't need that option, don't want to use it, you
don't have to : that's why it's an option. Personaly, I need
to use it, so I did it...

> an extension that easily maps to a type (for example .cgi or a URL then
> ends with a /) then Swish will still need to make the request to see if the
> document is indexable; just like it does now!

-sigh- You don't get it... If there is file that I _know_ I will
_never ever_ want to index just because of their extension,
why shouldn't I be free to let SWISH know about it by using
a configuration directive ????

> I believe that the fundamental disagreement here is whether the extension
> of a URL positively identifies the type of the document.

No, not at all. But I say that in some case, the extension
is far enough to positively exclude some files. It may be
wrong in some case, but I am willing to lost a few documents
if someone was weird enough to configure that ".jpg" contain
in reality an "html" page on his server... if that make me
gain 99% of speed !

Especially with the HTTP method, the largest amount of time is
lost in connecting to the remote server and transfering data from
it. So it should be avoid at all cost ! And I tell you that
it makes an awfully big differance. Big enough in many case
to make such indexing usable when it was totaly unusable...

I don't call such a gain of speed an insignificant optimization...

Yann Stettler

TheNet - Internet Services AG              CohProg SaRL                           
Anime and Manga Services         
Received on Tue Jan 19 12:58:11 1999