Skip to main content.
home | support | download

Back to List Archive

Re: User-Agent: Mozilla/2.0 and Mozilla/5.0 (compatible

From: Bill Moseley <moseley(at)>
Date: Wed Jan 10 2007 - 15:31:45 GMT
On Wed, Jan 10, 2007 at 07:14:01AM -0800, James wrote:
> So, you think that it is a case of them just putting that information
> into their user / server agent string to trick the server into
> thinking they are viewing through a browser?  That's an interesting
> thought.  So, maybe they aren't running any spiders through browsers.
> Or is it still possible that they do run the spider through a browser?

The user agent string is just sent with a request.  The server can
decided what to do with it -- the vast majority ignore it.  That's
what I'd recommend doing, too.

Do you have an actual problem with this?  I mean, do you know for a
*fact* (not because you guess so) that some server is not sending you
a document based on the user agent string?  Have you discussed this
with that site's admin?

Even if you do have a real problem, then just set the agent string.  I
think we covered how to do that now.

Bill Moseley

Unsubscribe from or help with the swish-e list:

Help with Swish-e:
Received on Wed Jan 10 07:31:47 2007