Skip to main content.
home | support | download

Back to List Archive

Running out of memory...

From: CheHax <chehax(at)not-real.gmail.com>
Date: Tue Apr 12 2005 - 08:32:27 GMT
Hello Swish-e list,

We are now trying to index our database datas using Swish-e and we
have run into the following error:

---
Indexing Data Source: "External-Program"
Indexing "./pl/full.pl"
External Program found: ./pl/full.pl
err: Ran out of memory (could not allocate 262144 more bytes)!
---

It happened after 14 hours of indexing.
The problem is that we have 4GB of memory, but it happened before this
hard limit was reached, as the system (windows server 2003) shows that
the memory usage caped at 2.6GB. We have defined a max swap size of
4GB so programs can use up to 8GB minus memory used by the system.

We are trying to index 23 millions records, which are all quite small
since we are using meta tags only (and group of meta tags).

We launched Swish-e with the -e directive so that it would use hard
drives instead of memory. Also note that temporary files were 730MB
big for properties and 450MB for idx.temp file when the program
crashed.

Any advice/help would be greatly appreciated. Thanks

Chehax
Received on Tue Apr 12 01:32:35 2005