Skip to main content.
home | support | download

Back to List Archive

Re:

From: Bill Pavich <BPavich(at)not-real.nctimes.com>
Date: Mon Jun 21 2004 - 22:35:33 GMT
Hello Peter.
Thanks for the response.
No, they are not shell errors, they are errors from the swish-e binary.
I think it might be a stdio limit with swish-e maybe? Maybe sfio would
help? Both my hard and soft open files descriptor limit on my sun box is
set to 16384. It's not a "real" too many open files problem with the OS.

Until swish-e implements real merging, your work around is perfect. I
should have thought of that. Thanks a lot for speaking up!

-----Original Message-----
From: swish-e@sunsite3.berkeley.edu
[mailto:swish-e@sunsite3.berkeley.edu] On Behalf Of Peter Karman
Sent: Monday, June 21, 2004 3:28 PM
To: Multiple recipients of list
Subject: [SWISH-E] Re: 


I do something similar, though because of how I am merging, I get shell 
errors about too many open files. I wasn't clear from your post whether 
you error was from swish or the shell. Anyway, to get around it, I group

them into bunches of 20, merge the 20, then merge all the 20index files 
into a master. Kind of round about, but it works.


Bill Pavich wrote on 6/21/04 5:01 PM:

> Hello everyone. Having a merge problem here.
> Running swish-e 2.4.1 on a Solaris (sparc) 2.9 box. Everything is 
> working fine, until now.=20 I use swish-e to create an index of pdf 
> files each night that were created that previous day. I store each 
> day's index files in a folder on disk. I allow folks to search these 
> daily indexes. But, I have been creating a merged "master" index of 
> all of these daily indexes each night using the -M merge index 
> feature. This allows folks to search the entire collection of daily 
> indexes instead of just being limited to one particular day. 
> Basically, it's a:
> 
> Swish-e -M /opt/vnews-index/vnews??????.index
> /opt/vnews-index/vnews-master.index
> 
> This has worked fine for the past couple of months, but I am now  up 
> to 130 individual index files and when the merge process runs, I get 
> the following error from swish-e
> 
> err: Index file error: Could not open the index file
> '/opt/vnews-index/vnews061804.index': Too many open files
> 
> Sounds like swish-e is running out of file handles? Does anyone have 
> any ideas? I can't just keep recreating a master index each night, 
> that would take increasingly too much time as days go by. Any ideas? 
> Is there a better way to accomplish what I'm trying to do?
> 
> Thanks.
> 
> 
> 
> 
> 
> *********************************************************************
> Due to deletion of content types excluded from this list by policy, 
> this multipart message was reduced to a single part, and from there to

> a plain text message.
> *********************************************************************

-- 
Peter Karman - Software Publications Engineer - Cray Inc
phone: 651-605-9009 - mailto:karman@cray.com
Received on Mon Jun 21 22:35:35 2004