Skip to main content.
home | support | download

Back to List Archive

Re: Swish-E with incremental index building

From: Peter Karman <karman(at)not-real.cray.com>
Date: Mon Nov 29 2004 - 21:38:55 GMT
Tilo Muetze wrote on 11/29/04 8:57 AM:

> So if I build Swish-E with --enable-incremental and later use -r to make 
> sure "old" documents gets removed from the index, is that what we need 
> to handle those huge amount of data, Or do you still see problems, 
> despite the fact that that 2.5. is just a BETA yet?
> 

What I hear you asking is, can I trust this incremental feature.

My understanding of the incremental feature is limited (Jose wrote it, 
so he could give a better answer), but my testing suggests this:

the incremental option works, but hasn't been stressed enough yet to 
really find all the bugs. I know of a couple minor ones related to 
ranking (they're on the developer to-do list). You are correct, I think, 
in thinking that the -r and -u options are the relevant ones for your 
project. I think if you do use it, you might consider yourself a "pilot 
tester" and let us know what you discover. :)

Even though the incremental feature has been available for a couple 
release cycles (including the soon-to-be-announced 2.4.3), it really 
needs more real-world exposure before the 'experimental' label is removed.

So try it out, stress it, see if it breaks. The more people who do that, 
the closer we can collectively come to calling it 'stable'.

-- 
Peter Karman  .  http://www.cray.com/craydoc/ .  karman(at)not-real.cray.com
Received on Mon Nov 29 13:38:56 2004