I've seen some threads about similar problems to the one I'm facing, yet
many were older solutions.
My base url is: http://library.princeton.edu . However, there are links to
other servers which I would want to index, without indexing the entire site.
Prior to indexing I have some knowledge of servers/directories, I do want to
For instance: I may want to index,
http://www.princeton.edu/~rbsc/exhibitions/online.html but not all of
www.princeton.edu. Or I may want to do
http://libweb5.princeton.edu/ejournals/by_title_zd.asp but not all of
Any thoughts or ideas? I'm using spider.pl with some configuration
Library Web Development Manager
Received on Thu Sep 16 12:09:18 2004