Skip to main content.
home | support | download

Back to List Archive

Re: Perl I/O buffering and the prog method

From: Thomas R. Bruce <trb2(at)>
Date: Fri Feb 04 2005 - 15:33:52 GMT
Bill Moseley wrote:

>On Fri, Feb 04, 2005 at 06:12:18AM -0800, Thomas R. Bruce wrote:
>>We run any number of custom spidering scripts that we use to feed 
>>swish-e, via the prog method.  They're in perl.  Recently, I've started 
>>messing around with turning buffering off on STDOUT for these scripts 
>>(via $|++) and my general impression, without formal benchmarks, is that 
>>they're running a good bit faster.  They're running on a Linux box with 
>>one of the 2.4.x kernels.
>What do you think that is?

I sorta suspect that it's an artifact of the Linux scheduling system, 
though I'm no kind of OS-diagnosing expert.  The reason I say this is 
that when one views the world through top with buffering turned off, 
both the swish-e and spidering processes remain fairly constantly near 
the top of the run list.  With buffering on,  the spidering process runs 
relatively constantly while swish-e comes and goes.  Of course, all this 
could be an artifact of the way top reports things, too.  I'm too 
clueless to know.

At the  moment I'm running  a fairly big job that can't be interrupted, 
or I'd try a compare-and-contrast test.  Will let you know once I have.


Thomas R. Bruce (
Director, Legal Information Institute
Cornell Law School

Due to deletion of content types excluded from this list by policy,
this multipart message was reduced to a single part, and from there
to a plain text message.
Received on Fri Feb 4 07:33:53 2005