On Wed, Feb 02, 2005 at 04:34:11PM -0500, Juan Carlos Avila / MTBASE wrote:

> I get the following output -- note the message "500 Chunked must be last 
> Transfer-Encoding 'chunked '" at the end. Also note that if I try to run 
> the spider just by changing the number at the end of tru URL (ie. 
> caso_numero=6897), it works fine!

That's odd.  What does the number have to do with the request????


> 
> spider.pl: Reading parameters from 'default'
> -- Starting to spider: 
> http://192.9.202.1/casos/VerCasoIdx?caso_numero=6896 --
> vvvvvvvvvvvvvvvv HEADERS for 
> http://192.9.202.1/casos/VerCasoIdx?caso_numero=6896 vvvvvvvvvvvvvvvvvvvvv

So, what's the difference between these two requests?  One is a HEAD
and the other is a GET, but same URL, right?  Sounds like a problem
with your servlet programmers....


> ---- Request ------
> HEAD http://192.9.202.1/casos/VerCasoIdx?caso_numero=6896
> Accept-Encoding: gzip; deflate
> From: swish@user.failed.to.set.email.invalid
> User-Agent: swish-e spider 2.2 http://swish-e.org/
> ---- Response ---
> Status: 200 OK
> Connection: Close
> Server: Jaguar Server Version 4.2
> Content-Length: 8192
> Content-Type: text/html
> Client-Date: Wed, 02 Feb 2005 21:28:50 GMT
> Client-Peer: 192.9.202.1:80
> Client-Response-Num: 2
> ^^^^^^^^^^^^^^^ END HEADERS ^^^^^^^^^^^^^^^^^^^^^^^^^^
> vvvvvvvvvvvvvvvv HEADERS for 
> http://192.9.202.1/casos/VerCasoIdx?caso_numero=6896 vvvvvvvvvvvvvvvvvvvvv
> ---- Request ------
> GET http://192.9.202.1/casos/VerCasoIdx?caso_numero=6896
> Accept-Encoding: gzip; deflate
> From: swish@user.failed.to.set.email.invalid
> User-Agent: swish-e spider 2.2 http://swish-e.org/
> ---- Response ---
> Status: 500 Chunked must be last Transfer-Encoding 'chunked '
> Content-Type: text/plain
> Client-Date: Wed, 02 Feb 2005 21:28:50 GMT
> Client-Warning: Internal response


Now, do you have netcat, ngrep, tcpdump or ethereal? The next step
would be to look at the actual tcp request to see *exactly* what is
being sent.  That "Request" is not an exact copy of what's sent to the
server.

I think LWP does add a Transfer-Encoding header:

In LWP/Protocol/http.pm:

   push(@h, "Transfer-Encoding" => "chunked");

But, again, I'd want to use ethereal to look to see exactly what's
going over the network.  That would be very interesting to look at.
Need more input, I guess is my response at this time....

Is your application making a second http request to another server?
Maybe that's where the error is coming from.

Try wiping your hard disk and installing Linux and Apache and see if
the problem goes away.  Well, try that as a last resort.






-- 
Bill Moseley
moseley@hank.org

Unsubscribe from or help with the swish-e list: 
   http://swish-e.org/Discussion/

Help with Swish-e:
   http://swish-e.org/current/docs
   swish-e@sunsite.berkeley.edu