Skip to main content.
home | support | download

Back to List Archive

RE: Server or documet limit on

From: <redna(at)>
Date: Fri Jan 09 2004 - 11:24:06 GMT
I've updated from a just downloaded version and the problem has
been fixed.

The matter was that when was making the summary for a server(a
normal server) it crashed.
Now it's allright.


>Come to think of it, you are probably right. I have not seen the issue in
>long time, and in the meanwhile I am sure I have updated LWP to the newest
>version. Anyway, my main point is that the spider definitely is not
>It is actually quite versatile.
>Aaron Bazar
>-----Original Message-----
>[]On Behalf Of Bill Moseley
>Sent: Wednesday, January 07, 2004 1:25 PM
>To: Multiple recipients of list
>Subject: [SWISH-E] RE: Server or documet limit on
>On Wed, Jan 07, 2004 at 10:14:02AM -0800, Aaron Bazar wrote:
>> FYI,
>> I have used to grab over 100,000 documents over a period of 16
>> hours. I have, at other times, run into some issues where
>> using up a huge chunk of memory. I never was able to determine why it
>> happened sometimes and not others.
>Maybe a different issue, but there was a problem a while back where the
>spider with some versions of LWP didn't free memory.  This was discussed
>on both the swish-e list and the LWP list.  Was it possible you were
>using different machines or different versions of LWP?
>Bill Moseley
Txat euskalduna >>>
Received on Fri Jan 9 11:24:18 2004