Skip to main content.
home | support | download

Back to List Archive

Re: a better way to index?

From: Hup Chen <hup(at)>
Date: Mon Mar 10 2003 - 19:43:59 GMT
> > Method 1:
> > index the whole 2M database
> > # $parse $source | $swishe -i stdin -S prog -f $pathOut/$test1 -c
> > swish.conf
> What was the max memory usage?

  The external parse program was using 2GB, swish-e took about 750MB RAM.

> > Method 1: took 3 hours 13 minutes
> Can you run that again with -e and report the time and max memory usage.
> As the hash tables fill indexing slows down.

  under -e mode, swish-e used only up to 150MB, but the external program
was still using 2GB.  The index time was about the same, 3:15.

Elapsed time: 03:15:27 CPU time: 00:-4:-3

> Nice to see merge is working fast.  It was not that long ago that it was
> very slow and memory intensive.

  I am glad to find swish-e.  Great search performance, reasonable index
time, cool price (some commercial product quoted me $50K-100K/per CPU).

> BTW -- nice hardware. ;)
  Thanks :)

Technical Manager

Search.  Compare.  Save.  (new book search)  (used book search)  (CDs)  (DVD & VHS)  (magazine subscriptions)
Received on Mon Mar 10 19:50:14 2003