> On Sat, 2002-07-13 at 17:39, Michael wrote:
> > proposed enhancements to swishspider
> > 1) add version number
> 
> You know, it might not be a bad idea to add the RCS Id.  I'll do
> that right now...
> 
> > 2) add call for HTML::Parser 3.00 as minimum support level
> 
> I just committed this to CVS.
> 
> > 3) add detection and escape for BAD URLS

A word of explaination. The reason I added this was that there was a 
URL within a web site that gave some problems. Specifically, the 
spider got stuck there, don't remember why. I did want to spider the 
website, just skip that particular part of it. There may be a more 
clever way to do it and perhaps amended match criteria like case 
insensitivity, etc.... would be better. Just some thoughts.

Michael

> 
> Good idea.  Perhaps Bill can comment on that when he has a chance. 
> :-)
> 
> Thanks for the info and suggestions!
> 
> -- 
>  David Norris
>   Dave's Web - http://www.webaugur.com/dave/
>   Augury Net - http://augur.homeip.net/
>   ICQ - 412039
>