Thank you for your reply!
I tested it again today. It shows that the crawler can only index the
webpages within "http://digital.lib.lehigh.edu". It cannot crawl the pages
on "rust.cc.lib.lehigh.edu" or any other websites, even though i used real
URLs instead of queries.
Any ideas about it?
Thank you very much!
On Wed, Mar 25, 2009 at 6:23 PM, David Norris <firstname.lastname@example.org> wrote:
> 2009/3/25 Zhou Xiang <email@example.com>:
> > Any ideas as to why these pages are not being indexed?
> I don't believe the old spider method works with queries. You would
> likely want to create a filter-based spider script that understands
> your query syntax and translates it to something useful you can later
> use in your search frontend.
> Or, alternatively, rewrite your entire website to use real URLs
> instead of queries.
> David L Norris
> Users mailing list
Users mailing list
Received on Thu Mar 26 16:29:54 2009