Skip to main content.
home | support | download

Back to List Archive

Re: Last part of merge very slow

From: <kopplin(at)not-real.technoir.nu>
Date: Wed Mar 07 2001 - 00:18:07 GMT
On Mon, 5 Mar 2001 jmruiz@boe.es wrote:

> Sounds strange... After "Writing MetaNames..." the work is almost 
> finished. Only a couple of fclose and lots of calls to free the used 
> memory are the remaining work but this should be fast unless
> your box is paging. Paging can make swish very slow and, at the 
> end of the proccess, it begins to release the used memory, so 
> paging can be the problem.

Ok, it looks like this is the problem afterall. I added another
128MB making 320MB total, and everything works fine until my
index size gets to about 30MB. At this point, with about 30MB
still free, it starts to use swap just a little bit, a few MB
maybe, but apparently this is enough to slow it to a crawl. I
didn't realize it would start using swap with that much free mem
left, or that merging seems to need about 10x final index size
in memory. At this rate, in 5 or 6 months I'll need a new box as
the current one only supports 512MB.
 
> The alpha release 2.1-dev20 has a special run mode (-e) that saves 
> RAM using temporal files but merge, in this version, is not yet 
> stable.

Is there a not quite so bleeding edge version you would
recommend?

Regards,
Mike Kopplin
Received on Wed Mar 7 00:24:10 2001