Skip to main content.
home | support | download

Back to List Archive

Re: Re: spider generates 'Wrong method or server' errors

From: Ron Klatchko <ron(at)not-real.ckm.ucsf.edu>
Date: Wed Oct 28 1998 - 17:44:17 GMT
David Norris wrote:
> Mostly being curious.  It's just something that popped into my mind when I
> read the mail.  If I understand your use of equivalent, there may be some
> cases where a web would exist as a sub directory as well as under its own
> domain name.  For example, http://myvirtualserver/ would be equivalent to
> http://myhostserver/myuserdir/
> This is how HTTP/1.1 stays semi-compatible with HTTP/1.0 browsers.  If there
> is no Request URI header, then a non-virtual URI is served.  Am I
> misunderstanding your use of equivalent?  Or, would you expect someone to
> use another method which I am currently overlooking?

The use of equivalent as currently implemented handles two cases:

A single server has multiple DNS names to access it.  Furthermore, links
use the different DNS names.  I didn't want to consider multiple DNS
names equivalent simply because they have the same IP (because of the
virtual server issue), so I decided to allow the user to tell me what
DNS names should be considered identical.

Indexing multiple servers.  Another common case is multiple servers
acting as a single logical server.  Examples that I have seen include
using one server for static requests and another server for dynamic
requests.

The reason that I require the method was that I was hoping to get a
solution that could eventually index different server types (ftp: would
be the next).  No plans on implementing that in the immediate future,
but hopefully one of these days.

moo

> 
> > > BTW, I think I have narrowed the spidering problem down to a
> > few lines of
> > > code.  I will have more info later.  It has to do with the
> > screwy command
> > > interpreter on Windows 9x.  NT would likely work fine.  To run the PERL
> > > script on 9x, one would have to call "start swishspider.pl...",
> > instead of
> > > "swishspider.pl..."
> >
> > I'm not sure if that's the correct solution.  At least from the command
> > line, start does not wait for the program to end before returning
> > control.  This would definitely break swish.
> 
> Hmmm, yes, that would be a problem.  Start.exe being a stub had slipped my
> mind.  Albeit bloated, somewhat complex, and several other things, I suppose
> one could link perl.lib into SWISH to make it directly understand the
> swishspider.pl script.  I would call it overboard, given this would only add
> Win9x support.  Perhaps, a simplified C replacement of the PERL script just
> for Win9x.  Just enough to be useful for simple projects.  If someone needed
> the full functionality, they can always use a decent Server platform that
> supports shell scripts.
> 
> ,David Norris
> 
> World Wide Web - http://www.geocities.com/CapeCanaveral/Lab/1652/
> Illusionary Web - http://illusionary.dyn.ml.org/ <-- 02:00 - 10:00 GMT
> Video/Audio Phone - callto:illusionary.dyn.ml.org
> Page via mail - 412039@pager.mirabilis.com
> ICQ Universal Internet Number - 412039
> E-Mail - kg9ae@geocities.com

-- 
moo
----------------------------------------------------------------------
          Ron Klatchko - Manager, Advanced Technology Group           
           UCSF Library and Center for Knowledge Management           
                           ron@ckm.ucsf.edu
Received on Wed Oct 28 09:56:50 1998