Im trying to index lots of websites. Im spidering with with "spider.pl".
(swish-e -S prog -c swish-e.conf)
To make life easy i have an include directive in my swish-e.conf:
And in sites.txt:
SwishProgParameters default http://bim.ce.kth.se http://www.svktf.se
etc... etc... (lots of sites)
Everything works great until i add "to many" sites, then i get this
Bad directive on line #2 of file sites.txt: /www.euronom.se http://www.e
urovac.nu etc... etc...
Seems to me there is a 2000 character limit. As soon as the included
"sites.txt" file grows over this limit, spider.pl barfs on me.
Am i missing something ?
Received on Sun Nov 17 17:43:42 2002