Skip to main content.
home | support | download

Back to List Archive

Re: [SWISH-E:331] Re: A new search engine for swish-e

From: J. J. Merelo <jmerelo(at)not-real.kal-el.ugr.es>
Date: Fri Jun 19 1998 - 20:02:18 GMT
Paul J. Lucas wrote:

> On Fri, 19 Jun 1998, Michael Ingvarsson wrote:
>
> > The degree project consisted in creating a search engine that keeps
> the
> > information, swish-e 1.1 indices, in memory all the time to speed up
> the
> > search by several orders of magnitude!
>
>         Uhm...it's the indexing that takes all the time, not the
>         searching.  SWISH-E is plenty fast enough for searching.  Or
> is
>         there something I'm missing about why you're doing what you're
>
>         doing?

Of course it is, but if you4ve got several 10-MB files, you4ll have to
load them each time you do a search (well, even if they4re not 10-MB in
size). The java servlet and associated RMI program keeps all the indexes
in memory, all the time; so that what you speed up is not really search,
but the overhead associated to loading indices each time.

>         P.S. FYI: SWISH++, by virtue of mmap(2), effectively does load
>
>         the entire index into memory increasing the speed of results,
>         but it's mostly because I'm lazy and doing the code that way
> is
>         easier than parsing the index file.

I know, I use it and it4s pretty fast, but still with big indices, or
with small indices but lots of request, and being CGIs as they are, you
could end up, as it4s happened to us, with 10 or 12 copies of the index
in memory.
See ya!

J
Received on Fri Jun 19 13:32:58 1998