Skip to main content.
home | support | download

Back to List Archive

Re: Bad links in documentation

From: Greg Fenton <greg_fenton(at)>
Date: Thu Jan 16 2003 - 03:31:58 GMT
--- John Almberg <> wrote:
> So the search engine has to index lots of static pages and one
> dynamic page.

I would suggest using "-S prog" and crawling with

It is quite straightforward to set up and works quite well.  I use the
spider because we use SSIs.  It adds a small amount of overhead to the
build, but our site (1100 pages or so) takes less than 4 minutes to
crawl and build (in fact, it might be less than 3 minutes...its been a
while since I timed it).

Hope this helps,

P.S. John, sorry about my previous reply...hit the wrong button :-)

Greg Fenton

Do you Yahoo!?
Yahoo! Mail Plus - Powerful. Affordable. Sign up now.
Received on Thu Jan 16 03:32:07 2003