Skip to main content.
home | support | download

Back to List Archive

RE: Getting the right files indexed the right way

From: Rob de Santos AFANA <rdesantos(at)not-real.afana.com>
Date: Thu Jan 29 2004 - 04:07:55 GMT
David, 
 
> > Thanks.  That makes a lot of sense from a security standpoint.   So,
> > what would be the best solution to get those filenames indexed by 
> > swish-e?
> 
> Is there any reason you do not index the gallery itself?  
> That seems more useful than indexing only the JPEG files.
>   http://www.afana.com/gallery/

At this point that seems to be the only practical solution.  
 
> You can have SWISH-E ignore the robots.txt file with 
> ignore_robots_file in SwishSpiderConfig.

Yes, but it simplifies things considerably to use it (robots.txt).  In
this case, I'll just have to let swish-e have a go at the gallery itself
and see how that goes.  Then, I'll just watch and see what the other
spiders on the net do. 

Regards,

-Rob de Santos
-Columbus, Ohio USA
Chairman of the Board,
Australian Football Association of North America (AFANA)
ph: 1-888-4AFANA1 (North America) (1-888-423-2621)
ph: 1-614-338-0002 (outside NA)  
e-mail: rdesantos(at)not-real.afana.com   web: <http://www.afana.com>
Received on Wed Jan 28 20:07:56 2004