Skip to main content.
home | support | download

Back to List Archive

RE: Server or documet limit on spider.pl

From: Bill Moseley <moseley(at)not-real.hank.org>
Date: Mon Jan 12 2004 - 17:44:44 GMT
On Mon, Jan 12, 2004 at 08:54:04AM -0800, Ander wrote:
> I got it again,
> This is the context:
> 
> [ander@localhost prog-bin]$ ./spider.pl.bak SwishSpiderConfig.pl > /dev/null
> ./spider.pl.bak: Reading parameters from 'SwishSpiderConfig.pl'
> Use of uninitialized value in printf at ./spider.pl.bak line 282.
> Modification of a read-only value attempted at ./spider.pl.bak line 282.
> 
> 
> And this is the 282 line of spider.pl.bak:
> 
> 
> 282: printf STDERR "\nSummary for: $server->{base_url}\n";
> 283:
> 284:    for ( sort keys %{$server->{counts}} ) {
> 285:        printf STDERR "%${max_width}s: %${max_num}s  (%0.1f/sec)\n",
> 286:        $_,
> 287:        commify( $server->{counts}{$_} ),
> 288:        $server->{counts}{$_}/$start;
> 289:    }

Odd.  First, that's the code that gives the summary at the end of 
spidering, but I thought the problem was the spider was stopping before 
it had spidered everything.

Second, I don't see anything odd about line 282 -- there's not 
modification of any variables there.

Are you running that buggy Perl 5.6.0 by chance?

Are you doing anything tricky in your SwishSpiderConfig.pl file?

-- 
Bill Moseley
moseley@hank.org
Received on Mon Jan 12 17:44:55 2004