headers
Nico de Jager | 12 Dec 2007 12:34

CHUNGA:INPUT-CHUNKING-UNEXPECTED-END-OF-FILE

Hi

I have a fingerprint scanning application, that uploads fingerprint images using Drakma to a Hunchentoot
server. Everything works great as long as I don't use a secure connection. With https, I get a
CHUNGA:INPUT-CHUNKING-UNEXPECTED-END-OF-FILE condition on the server, and "Timeout while writing
8192 bytes to #<COMM:SOCKET-STREAM >" on the client. Sometimes the uploads are successful with https, though.

The server runs LispWorks 5.0.2 on Debian Etch. The client runs on LispWorks 5.0.2 on Windows XP SP2. Part of
the application pulls in the trivial-ldap module which depends on cl+ssl - not sure if this can be an issue
as cl+ssl is not needed on LispWorks for Drakma and Hunchentoot.

I am sure I am doing something stupid. The handler that does the actual upload on the client (which also uses
Hunchentoot) is:
(define-easy-handler (upload-fp-image.action :uri
"/hunchentoot/patient-study-db/upload-fp-image.action") ()
  (let ((cookie-jar (make-instance 'drakma:cookie-jar
                                   :cookies (list (make-instance 'drakma:cookie :name (session-value :hs-name) :value (session-value :hs-id)
                                                                 :path (session-value :hs-path) :domain (session-value :hs-domain))))))
    (handler-case
        (let ((html-result (drakma:http-request (session-value :upload-url)
                                                :method :post
                                                :parameters (list (cons "finger" (session-value :finger))
                                                                  (cons "img-file" (pathname *scanned-file*))
                                                                  (cons "hs-id" (session-value :hs-id)))
                                                :user-agent (session-value :user-agent)
                                                :cookie-jar cookie-jar)))
          (delete-file *scanned-file*)
          (with-html-output-to-string (*standard-output*)
            (str html-result)))
      (condition (con)
(Continue reading)

Permalink | Reply | 4 comments |
headers
Nico de Jager | 12 Dec 2007 14:59

Re: CHUNGA:INPUT-CHUNKING-UNEXPECTED-END-OF-FILE

Nico de Jager <ndj <at> hivsa.com> writes:

> Hi
>
> I have a fingerprint scanning application, that uploads fingerprint images using Drakma to a
Hunchentoot server. Everything works great as long as I don't use a secure connection. With https, I get a
CHUNGA:INPUT-CHUNKING-UNEXPECTED-END-OF-FILE condition on the server, and "Timeout while writing
8192 bytes to #<COMM:SOCKET-STREAM >" on the client. Sometimes the uploads are successful with https, though.
>
> The server runs LispWorks 5.0.2 on Debian Etch. The client runs on LispWorks 5.0.2 on Windows XP SP2. Part
of the application pulls in the trivial-ldap module which depends on cl+ssl - not sure if this can be an
issue as cl+ssl is not needed on LispWorks for Drakma and Hunchentoot.

Trying to isolate the problem, I set up a general upload easy handlers in a separate application. All https
uploads work fine from Linux clients but not from Windows, so I guess it is my version of OpenSSL on the
Windows box. I use a binary version which I downloaded here (Win32 OpenSSL v0.9.8g):
http://www.slproweb.com/products/Win32OpenSSL.html. I followed this link from the FAQ on www.openssl.org.

Does anyone have advice on easily getting a reliable/working copy of OpenSSL on Windows (preferably a
binary package), before I compile my own? Or is it something else?

Thanks.
Nico

>
> I am sure I am doing something stupid. The handler that does the actual upload on the client (which also uses
Hunchentoot) is:
> (define-easy-handler (upload-fp-image.action :uri
"/hunchentoot/patient-study-db/upload-fp-image.action") ()
>   (let ((cookie-jar (make-instance 'drakma:cookie-jar
(Continue reading)

Permalink | Reply | 1 comment |
headers
Edi Weitz | 13 Dec 2007 21:36
Picon
Favicon

Re: CHUNGA:INPUT-CHUNKING-UNEXPECTED-END-OF-FILE

On Wed, 12 Dec 2007 13:34:14 +0200, Nico de Jager <ndj <at> hivsa.com> wrote:

> I have a fingerprint scanning application, that uploads fingerprint
> images using Drakma to a Hunchentoot server. Everything works great
> as long as I don't use a secure connection. With https, I get a
> CHUNGA:INPUT-CHUNKING-UNEXPECTED-END-OF-FILE condition on the
> server, and "Timeout while writing 8192 bytes to
> #<COMM:SOCKET-STREAM >" on the client. Sometimes the uploads are
> successful with https, though.

It's hard to say something about this from here.  Are you able to
generate a small, self-contained example that exposes the problem?
What did you do to isolate the error?  Does it work without the
trivial-ldap module being loaded?  Does it work with another Lisp
implementation on the client or on the server side?  Does it work if
you use :CONTENT-LENGTH T in your Drakma request?

Edi.
Permalink | Reply | 1 comment |
headers
Edi Weitz | 13 Dec 2007 21:37
Picon
Favicon

Re: CHUNGA:INPUT-CHUNKING-UNEXPECTED-END-OF-FILE

On Wed, 12 Dec 2007 15:59:20 +0200, Nico de Jager <ndj <at> hivsa.com> wrote:

> Trying to isolate the problem, I set up a general upload easy
> handlers in a separate application. All https uploads work fine from
> Linux clients but not from Windows, so I guess it is my version of
> OpenSSL on the Windows box. I use a binary version which I
> downloaded here (Win32 OpenSSL v0.9.8g):
> http://www.slproweb.com/products/Win32OpenSSL.html. I followed this
> link from the FAQ on www.openssl.org.
>
> Does anyone have advice on easily getting a reliable/working copy of
> OpenSSL on Windows (preferably a binary package), before I compile
> my own? Or is it something else?

I use the one you also have.  They are the ones recommended by
LispWorks:

  http://www.lispworks.com/documentation/lw50/LWUG/html/lwuser-344.htm
Permalink | Reply | 0 comments |
headers
Peter Eddy | 17 Dec 2007 18:45
Picon

question about encodings and content length calculation

Hi,

I'm trying resolve a problem I'm having trying to send utf-8 data and
the calculation of the length of that data. If I make the following
post:

(defconstant +utf-8+ (make-external-format :utf-8 :eol-style :lf)

(http-request "http://odeon:5984"
                       :content-type "application/xml"
                       :external-format-out +utf-8+
                       :content "e"
                       :method :put)

The resulting request looks like this:

PUT / HTTP/1.1
Host: odeon:5984
User-Agent: Drakma/0.11.1 (OpenMCL Version 1.1-pre-070722
(DarwinX8664); Darwin; 9.1.0; http://weitz.de/drakma/)
Accept: */*
Connection: close
Content-Type: application/xml
Content-Length: 1

e

Which is right. But if I replace the "e" content with "é"  (e accent
acute), the request looks like this:
(Continue reading)

Permalink | Reply | 2 comments |
headers
Edi Weitz | 17 Dec 2007 21:25
Picon
Favicon

Re: question about encodings and content length calculation

On Mon, 17 Dec 2007 12:45:51 -0500, "Peter Eddy" <peter.eddy <at> gmail.com> wrote:

> Should I be calculating the length myself?

Yes.  See here:

  http://weitz.de/drakma/#content-length

One could argue that Drakma should be able to compute the right value
itself, but this is more trickier than one first thinks.  Of course,
if you think you know how to do this in a way that covers all corner
cases, I'd be happy to accept a patch.  The general design philosophy
of Drakma is that it should be as easy to use as possible.

  http://weitz.de/patches.html

Thanks,
Edi.
Permalink | Reply | 1 comment |
headers
Peter Eddy | 17 Dec 2007 22:53
Picon

Re: question about encodings and content length calculation

On Dec 17, 2007 3:25 PM, Edi Weitz <edi <at> agharta.de> wrote:
> On Mon, 17 Dec 2007 12:45:51 -0500, "Peter Eddy" <peter.eddy <at> gmail.com> wrote:
>
> > Should I be calculating the length myself?
>
> Yes.  See here:
>
>   http://weitz.de/drakma/#content-length
>
> One could argue that Drakma should be able to compute the right value
> itself, but this is more trickier than one first thinks.  Of course,
> if you think you know how to do this in a way that covers all corner
> cases, I'd be happy to accept a patch.  The general design philosophy
> of Drakma is that it should be as easy to use as possible.

It turns out that in my case setting content-length to nil (and
therefore using chunked encoding) works perfectly. I was sure I'd
tried that before, but must not have. Anyway, I think that qualifies
as easy.

Thanks!

- Peter
Permalink | Reply | 0 comments |
headers
Edi Weitz | 22 Dec 2007 03:29
Picon
Favicon

Re: [patch] drakma, support for connection-timeout on SBCL

On Fri, 21 Dec 2007 13:58:40 +0100, Stanislaw Halik <sthalik <at> tehran.lain.pl> wrote:

> I attach the patch for drakma to support the `connection-timeout'
> argument for `http-request' on SBCL.  Hope you like it.

Hi Stanislaw,

I'm copying my reply to the Drakma mailing list where I think this
discussion should take place.  Thank you for the patches you sent, but
I think that this functionality should rather be in usocket and not in
Drakma.  If it's offered by usocket, then Drakma will use it.  Have
you considered sending a patch to the usocket maintainers?

Thanks again,
Edi.
Permalink | Reply | 0 comments |
headers
Stanislaw Halik | 23 Dec 2007 05:34
Picon

Re: [patch] drakma, support for connection-timeout on SBCL

I've updated the patch to support the IO-TIMEOUT keyword argument. It's
not possible to non-intrusively support read- and write-timeouts
separately to bring the API in congruency with LispWorks.

The resolver error gets changed to SIMPLE-ERROR because it's not an
ERROR, but a CONDITION (signalled with ERROR, though).

I've stress-tested the patch, running 30 threads doing HTTP-REQUEST.

My rationale for including the patch - it doesn't change the program
logic much, and except for the function definition and socket connection
there's no special-casing for SBCL.

The timeout option is crucial for me: my typical usage is running
concurrent threads accessing URLs posted in spam to make the senders
think I'm a used accessing the page, in turn sending more spam,
addresses of which land in my spamtrap. Some of the URLs point to broken
httpds, leaving an open connection without sending any replies. This
leads to unfinished requests using threads and hanging for days.

If you feel like maintaing a piece of code for SBCL, by all means please
do so. I think other SBCL users might benefit from it.

--

-- 
   /\   
    /   Jabber ID       :: sthalik <at> jabber.autocom.pl
  \ \/  Unix stuff      :: http://tehran.lain.pl
 \/\    Yet Another RBL :: http://rbl.lain.pl
Attachment (drakma-sbcl-connection-timeout.diff): text/x-diff, 3385 bytes
(Continue reading)

Permalink | Reply | 502 comments |
headers
Stanislaw Halik | 23 Dec 2007 06:13
Picon

Re: [patch] drakma, support for connection-timeout on SBCL

On Sun, Dec 23, 2007, Stanislaw Halik wrote:
> [...]

As for the reason I haven't submitted the patch to usocket itself:
usocket is a compatibility layer, comprising of features readily
available in most of CL implementations. With socket timeouts supported
so far by only for LispWorks and SBCL, I believe the patch would have no
chances of getting accepted.

--

-- 
   /\   
    /   Jabber ID       :: sthalik <at> jabber.autocom.pl
  \ \/  Unix stuff      :: http://tehran.lain.pl
 \/\    Yet Another RBL :: http://rbl.lain.pl
Permalink | Reply | 502 comments |

Gmane