Skip to main content.
home | support | download

Back to List Archive

Re: Filter.pm & Windows Thread safety

From: Bill Moseley <moseley(at)not-real.hank.org>
Date: Mon May 24 2004 - 17:55:09 GMT
On Mon, May 24, 2004 at 10:32:51AM -0700, Job, James wrote:
> This link may explain the concern better:
> http://www.xav.com/perl/lib/IPC/Open2.html 
> Basically, it appears that open2() does NOT WAIT for, nor reap the child
> process after it terminates.

There must be a better way to run external processes under Windows that
using IPC::Open2.

Anyway, my concern is that you are waiting for the process to finish
before reading any data from the process.  Maybe I'm not understanding
IPC well enough, but doesn't it seem like the process may not finish if
data is not read from the pipe?

So, I would think it better in windows_fork() to return an object that
contains the file handle and when that object goes out of scope then
reap the child process (since IPC::Open2 doesn't reap on close).  That
make sense to you?

But again, there's probably a better way to do this under windows.
That's a question I've been asking for, oh, about four years.  Might be
a fun google search to see how long I've been asking around about how to
open external piped programs securely on Windows...

> My ZLIB problem is on hold for the moment.  I shifted from Suse Enterprise
> for AMD64 to Win2003/32bit in order to get SWISH-E up and running quickly
> (have a deadline to meet), but I won't forget about the 64bit/Linux Opterons
> though...  Fortunately, I've got a couple Opterons set aside for test & dev,
> so I can return to it when I have time.

Ok, post again when you have time to test.

> I seem to remember seeing some discussion in the archives about filter
> processes stalling in the past (only getting 64 .doc(s) or something like
> that).  People experiencing those problems may want to try this hack.  I had
> a similar result, only getting 23 docs and 55 pdfs indexed from my stock
> crawls.  I now get 55 docs and 335 pdfs (and no skipped due to filter
> warnings).

I think it makes sense.  My problem is I don't know what to tell people
when trying to debug something like this on Windows.  On Linux 64 catdoc
process might be obvious -- is there a way to do a process tree listing
on Windows?

Thanks for the follow up.

-- 
Bill Moseley
moseley@hank.org
Received on Mon May 24 10:55:09 2004