I set up two servers with dev-25 today.

The first one went great, no problems at all, indexed the site (only about
700 files) in about 30 seconds, even on an ancient P133 server.

The second one, though ... running on an Alpha/533 with 320 MB, indexed
about 90,000 files, and memory usage (using -e) was about 30 MB while
swish-e was reading the files.  Memory usage jumped up to about 90 MB once
the common word removal started.

Still, things looked to be OK, but something got stuck when it got to
writing index entries.  CPU usage stayed maxed out.  The files were not
being updated, and yet the process stayed running.  I finally killed it an
hour later.  swish-e output follows.


Checking dir "/local/web/articles/96"...
Checking dir "/local/web/articles/97"...
Checking dir "/local/web/articles/98"...
Checking dir "/local/web/articles/99"...
Checking dir "/local/web/articles/preview"...

Removing very common words...
  Getting IgnoreLimit stopwords: Complete                            
10 words removed by IgnoreLimit:
1, 2, i, s, 10, driver, racing, time, race, car, 
Writing main index...
Sorting words ...
Sorting 242541 words alphabetically
Writing header ...
Writing index entries ...
  Writing word text: Complete
  Writing word hash: Complete
  Writing word data: ...^Z
[1] + Stopped              nice -20 swish-e -e -c /usr/local/swish/article
stratos 176 # ls -l
total 196621
-rw-------  1 root  wheel  201218792 May 18 21:27 swtmploc03118a
-rw-------  1 root  wheel      12492 May 18 21:28 swtmploc03118b
stratos 177 # ls -l /usr/local/swish
total 9611
-rw-rw-r--  1 tomh  wheel     7967 May 18 20:54 articles.cfg
-rw-rw-r--  1 root  wheel  4163411 May 18 21:27 articles.index.prop.temp
-rw-rw-r--  1 root  wheel  9675275 May 18 21:29 articles.index.temp
-rw-r--r--  1 root  wheel     2267 May 18 20:55 stopwords.txt
stratos 178 # fg
nice -20 swish-e -e -c /usr/local/swish/article
^C
stratos 179 # 


And from top:

  PID USERNAME PRI NICE   SIZE   RES STATE   TIME   WCPU    CPU COMMAND
 3118 root     103   20    89M   82M run    86:43 95.36% 95.36% swish-e
21225 nobody    18    0  7808K 9816K sleep   0:00  1.41%  1.37% httpd
17628 nobody    18    0    10M   12M sleep   0:02  0.10%  0.10% httpd


Could I be running into some sort of a limit?

Tom Haapanen
tomh@motorsport.com