Skip to main content.
home | support | download

Back to List Archive

Re: Getting the right files indexed the right way

From: Bill Moseley <moseley(at)not-real.hank.org>
Date: Thu Jan 29 2004 - 02:47:10 GMT
On Wed, Jan 28, 2004 at 08:33:45PM -0500, Rob de Santos AFANA wrote:

> One other problem has cropped up.  We have a picture gallery application
> done in php ("Gallery") which allows users to view sets of pictures.
> The gallery directory itself is off limits to robots via robots.txt but
> the pics are in subdirectories of a directory called /album/.  That is
> open to spidering but for some reason swish-e doesn't index the
> subdirectories of /album/.  I tried this in the swish-e config file but
> it didn't work:
> -----
> SwishProgParameters
> /home/afana/public_html/swish-e/lib/swish-e/SwishSpiderConfig.pl
> http://www.afana.com http://www.afana.com/blog/archives/
> http://www.afana.com/album/
> 
> obeyRobotsNoIndex yes

That's for the -S http method of indexing so it doesn't apply to -S prog
with spider.pl.

Are there any links to follow in /album/?

-- 
Bill Moseley
moseley@hank.org
Received on Wed Jan 28 18:47:11 2004