How about using:
debug => DEBUG_URL | DEBUG_SKIPPED | DEBUG_FAILED | DEBUG_HEADERS,
within your @servers
On Jan 27, 2006, at 9:45 AM, andy rosbrook wrote:
> Well i just want to know after each URL in spider.config weather the
> spidering was a success or a failure. I know i could just check for a
> complete index.swish-e but this doesnt allow me to capture any error
> messages.
>
> Ill take a look at grabbin STDERR though, thanks.
>
>
>> From: Bill Moseley <moseley@hank.org>
>> Reply-To: moseley@hank.org
>> To: Multiple recipients of list <swish-e@sunsite3.berkeley.edu>
>> Subject: [SWISH-E] Re: Callback Functions For Indexing
>> Date: Fri, 27 Jan 2006 07:21:45 -0800 (PST)
>>
>> On Fri, Jan 27, 2006 at 06:30:56AM -0800, andy rosbrook wrote:
>>> Is there anyway to use a callback function to catch errors when
>> spidering
>>> websites with spider.pl?
>>>
>>> I am currently spidering only a few small sites at a time and
>>> need a way
>> of
>>> knowing weather the spider successfully indexed the site or not,
>>> is this
>>> possible? if so is there a way of grabbing the error message into
>>> perl?
>>
>> Not sure what you mean. You want to know if any file returned a
>> non-200 status? Or if swish-e indexed any words?
>>
>>
>> IPC::Open3 will capture stderr and stdout.
>>
>> --
>> Bill Moseley
>> moseley@hank.org
>>
>> Unsubscribe from or help with the swish-e list:
>> http://swish-e.org/Discussion/
>>
>> Help with Swish-e:
>> http://swish-e.org/current/docs
>> swish-e@sunsite.berkeley.edu
>>
>
> _________________________________________________________________
> Are you using the latest version of MSN Messenger? Download MSN
> Messenger
> 7.5 today! http://messenger.msn.co.uk
>
Bill
William M. Conlon, P.E., Ph.D.
To the Point
345 California Avenue Suite 2
Palo Alto, CA 94306
vox: 650.327.2175 (direct)
fax: 650.329.8335
mobile: 650.906.9929
e-mail: mailto:bill@tothept.com
web: http://www.tothept.com
Received on Fri Jan 27 10:24:18 2006