Skip to main content.
home | support | download

Back to List Archive

Re: Web Page listing software necessary when

From: Bill Moseley <moseley(at)not-real.hank.org>
Date: Tue Jun 12 2001 - 14:58:11 GMT
At 07:29 AM 06/12/01 -0700, AusAqua wrote:
>Thanks to Chris for putting together the summary of requirements for
>swish-e.  This is a most helpful thing for a newbie such as myself.

Just a comment.  Chris is using a version of a CGI script that I wrote that
required a bunch of modules.  After my dogma was run over by other
developers, there's now a "lite" version that doesn't require as much work
to install.

To run the spider.pl you will need to install modules, true, but most of
the same ones you would need for -S http method.

>"Do the modules need to be in the same directory as PERL (or Swish), in
>order to be accessible to PERL & Swish, or can they in fact be loaded
>anywhere within my web site and still be accessed as required by the web
>host's Perl program?"

No.  First, if you have shell access then you should be able to install
modules locally in your own directory (but not in "web space").  The perl
modules do not need to be located anywhere special.  You install the
modules using a PREFIX, and then add a 'use' line in the perl program.  So,
yes, swish and perl can be located anyplace as long as they can be seen by
a process running as the web server runs.


>Another approach on which I'd appreciate comment, might be to load swish,
>Perl, the modules described above, and a copy of my Web Site onto a Linux
>platform hard drive, then do the indexing on that drive, before FTPing the
>indexes and swish-e into appropriate directories on the hosted web site
>(also Linux platform).

The index can be copied around, but not swish-e unless you are sure you
share the same platform as the target system.

>Now to my question to the group:
>
>"Are the modules (described above) needed only at the time of swish's
>indexing of the web site, or are they also needed at such times as someone
>accesses the Web Page to do a "search" of the indexes on the site ?"

Most of those are for spidering.  And most are part of the LWP bundle of
modules.  LWP is widely used, and thus is commonly installed on machines,
so you may not need to install any modules.  It's needed for both the -S
http and -S prog (with spider.pl) methods.  The MD5 module is not really
needed.  The HTML:: parsing modules are needed to parse links from
documents.  None of this is needed if you are not spidering your web site,
and can use the -S fs method.

My experience is that although ISPs offer perl and many perl modules, it's
less likely to find an ISP that keeps things up to date.  I tend to keep my
own perl library, regardless of what the ISP offers.

Does that answer your question?



Bill Moseley
mailto:moseley@hank.org
Received on Tue Jun 12 15:17:20 2001