In case anyone else runs into this, yesterday I posted a problem about a
spider that didn't find any links on my index page even though there were
plenty there. I eventually (with the help of Giulia Hill and Ron Klatchko)
figured out that some meta tags in my header were causing the problem. The
solution was to modify the spider perl code so that the line that looked like
if( $response->header("content-type") eq "text/html" ) {
ended up looking like
if( substr($response->header("content-type"),0,9) eq "text/html" ) {
Now everything seems to work like a champ...
Thanks Ron and Giulia!
Bruce Bowler 207.633.9600 (voice)
Research Associate 207.633.9641 (fax)
Bigelow Laboratory for Ocean Sciences bbowler@bigelow.org
West Boothbay Harbor ME 04575 http://www.bigelow.org/
Received on Wed Jan 27 12:29:04 1999