Skip to main content.
home | support | download

Back to List Archive

limit in large IncludeConfigFile

From: Leif Larsson <leif.larsson(at)not-real.l3system.se>
Date: Sun Nov 17 2002 - 17:43:13 GMT
Hello,

Im trying to index lots of websites. Im spidering with with "spider.pl".
(swish-e -S prog -c swish-e.conf)
To make life easy i have an include directive in my swish-e.conf:
IncludeConfigFile sites.txt

And in sites.txt:
SwishProgParameters default http://bim.ce.kth.se http://www.svktf.se
etc... etc... (lots of sites)

Everything works great until i add "to many" sites, then i get this
error:
Bad directive on line #2 of file sites.txt: /www.euronom.se http://www.e
urovac.nu etc... etc...

Seems to me there is a 2000 character limit. As soon as the included
"sites.txt" file grows over this limit, spider.pl barfs on me.

Am i missing something ?

Thanks,

Leif
Received on Sun Nov 17 17:43:42 2002