Michael Minerva | 15 Mar 18:53

Changing content-type seems to have no effect

I have been trying to get my lisp application to interface and upload a file to my drupal 7 website using
Drakma.  

So far I have been able to to connect to my drupal site and authenticate and store the session cookie (thank
you Ryan Davis, I just found your response the other day).  Here is the code I use to do that:

(with-output-to-string (stream)
  (setf drakma:*header-stream* stream)
(let ((cookie-jar (make-instance 'drakma:cookie-jar)))
  (drakma:http-request
   "http://localhost:8888/?q=rest/user/login"
    :method :post
    :cookie-jar cookie-jar
    :parameters 
    '(("username" . "login")
      ("password" . "password"))
    )
  (drakma:http-request
   "http://localhost:8888/?q=rest/file"
    :method :post
    :cookie-jar cookie-jar
    :content-length t
    :parameters '(("file1" . #p"/Users/Mike/hello.png"));("file1"
#p"/Users/Mike/Desktop/mario-gif.gif" :content-type "image/gif" :filename "mario-gif.gif"))
    :content-type "application/x-www-form-urlencoded" 
    ;:parameters 
    )))

This code successfully sends both posts but the second post does not actually upload the file (but it does
return 200 OK, no errors).  I have successfully uploaded to by drupal site using PHP Curl and in that code, I
(Continue reading)

William Halliburton | 10 Mar 01:08
Picon

Unwanted url-encoding of GET parameters.


Hello folks,

I'm trying to use drakma to fetch urls that contain utf8 characters but HTTP-REQUEST automatically url encodes any non latin-1 ascii characters. 

On my cursory reading of the RFCs, this seems conforming behavor, but in this case it is definitely unwanted. 

For example, the following url


if entered directly into the browser correctly returns the text-to-speech audo file but 

when attempting to use HTTP-REQUEST, the url is being url encoded into


of which google does not url-decode and fails to return the correct data.

So, for this case, the url-encoding is unwanted. 

I am willing to submit patch an additional argument into HTTP-REQUEST to disallow the encoding.

Thoughts?

Thank you,
William
_______________________________________________
drakma-devel mailing list
drakma-devel <at> common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/drakma-devel
Robert Brown | 9 Mar 16:24
Picon

drakma and non-ASCII content

A co-worker of mine had some problems today using Drakma to POST a STRING
containing non-ASCII characters encoded as UTF-8.  He was doing something
equivalent to:

(http-request "http://zappa.com/favicon.ico"
              :method :post
              :content (concatenate 'string
                                    "hello" (string #\white_square) "world")
              :external-format-out :utf-8)

which ends up setting the Content-Length header value to 11, which is the
LENGTH in characters of the string being sent.

The documentation says that "if content is a sequence, Drakma will use LENGTH
to determine its length and will use the result for the 'Content-Length' header
sent to the server," so Drakma is working as documented.

Also, I think I understand why this behavior is the default.  You don't want to
scan a content string in order to determine what its length will be in octets
once it has been encoded.  My co-worker could have used a vector of type
(unsigned-byte 8) to hold his UTF-8 encoded data.

However, I think the current Drakma behavior may be a mistake.  People who want
high performance are probably manipulating encoded strings as vectors of
(unsigned-byte 8).  It's the casual users, those sending strings, who are the
ones most likely to be bitten by the default behavior, but only when their
strings contain non-ASCII characters.

bob
Willi Nesensohn | 27 Feb 20:20
Picon

http redirect following url-encodes location

I'm using drakma to GET data over http. It turns out that when following 
redirects, drakma uses the location in the redirection header, which it 
url-encodes. Usually this might not be a problem, but when the location 
redirected to is already url-encoded this method fails.

Is this intended behaviour?

I'm not sure what the clean solution to this would be, as decoding the 
location before using it just doesn't feel right. For the lack of a 
better solution that's what i'm doing right now though.

Any pointers in the right direction would be greatly appreciated.

Best regards,
Wilfried Nesensohn
Michael Minerva | 9 Feb 21:41

A simple Cookie and viewing the actual Post

I just started using drakma today and so far it seems really great but I had a couple quesitons.

First, is it possible to send a simpler for example i want my http-post to have something like this:

Cookie: SESSdc0685ed01f285dab628a3700259e6bc=rCfmWLUeugRQNAQTs14yox_DWqZyGE2fYUS8W24j4co

I tried making a drakma cookie and adding it to a cookie-jar but none of that seemed to work.

Also, is there a way to print the actual post I am sending so I can see what might be wrong with it?

Thanks a lot,

--Mike
Rob Blackwell | 14 Jan 13:15
Picon
Favicon
Gravatar

Drakma and client certificates?

Hi,

I'm trying to use client certificates with Drakma to call an existing web based API.

I have the latest code from https://github.com/edicl/drakma installed in my quicklisp local-projects
directory. I'm using SBCL on OS X and Linux.

The client certificate is a .pfx file, and I've converted it to .pem format using

openssl pkcs12 -in robblackwellmanage.pfx -out robblackwellmanage.pfx.pem

I have verfied that this works as a server certifcate, using hunchentoot, thus:

(hunchentoot:start (make-instance 'hunchentoot:ssl-acceptor :ssl-privatekey-file
"/Users/reb/certs/robblackwellmanage.pfx.pem" :ssl-certificate-file
"/Users/reb/certs/robblackwellmanage.pfx.pem" :ssl-privatekey-password "password" :port 4343))

When I try to use this as a client certificate:

 (drakma:http-request resource
			 :certificate "/Users/reb/certs/robblackwellmanage.pfx.pem"
			 :key "/Users/reb/certs/robblackwellmanage.pfx.pem"
			 :certificate-password "password"
			 :method :get
			 :content ""
			 :content-type "application/xml"
			 :additional-headers headers)

I get the following error

SSL initialization error: Can't load certificate passwordSSL error queue is empty.
   [Condition of type CL+SSL::SSL-ERROR-INITIALIZE]

Interestingly, if I deliberately supply the wrong password then I get a different error

SSL initialization error: Can't load RSA private key file
/Users/reb/certs/robblackwellmanage.pfx.pemSSL error queue is empty.
   [Condition of type CL+SSL::SSL-ERROR-INITIALIZE]

I guess I'm doing something really silly - please excuse my ignorance, but any advice, suggestions or
guidance on how to proceed would be very much appreciated.

Thanks!

Rob
mohegan-skunkworks | 29 Dec 19:01

cl-oauth and drakma interoperability issue

I just filled an issue on github...

The text is below...

Hi,
I'm using cl-oauth to authenticate with twitter (it's in the cl-twitter package, also on github).

I received a complaint from a quicklisp user that he couldn't authenticate with twitter.

After looking into this a bit I realized that drakma is encoding the uri on get requests. Now, cl-oauth passes in an encoded uri so this uri gets encoded twice. This was never a problem for me because I was using v 1.2.3 of drakma where this didn't happen.

Here's the code diff (plus my proposed change :)
========================== v 1.2.3 ==================================
(when (and (not parameters-used-p)
parameters)
(setf (uri-query uri)
;; append parameters to existing query of URI
(format nil "~ <at> [~A~]~:*~:[~;&~]~A"
(uri-query uri)
(alist-to-url-encoded-string parameters external-format-out))))

===========================current github/v1.2.4=============
(when-let (all-get-parameters
(append (dissect-query (uri-query uri))
(and (not parameters-used-p) parameters)))
(setf (uri-query uri)
(alist-to-url-encoded-string all-get-parameters external-format-out)))
===============================my proposed change===================================================
(when (and (not parameters-used-p)
parameters)
(when-let (all-get-parameters
(append (dissect-query (uri-query uri))
(and (not parameters-used-p) parameters)))
(setf (uri-query uri)
(alist-to-url-encoded-string all-get-parameters external-format-out))))

A few more comments :

This is the hand-off in cl-oauth :

(defun http-request (uri &key (request-method :get) parameters drakma-args)
  ;; TODO handle redirects properly
  (let* ((param-string-encoded (alist->query-string parameters :include-leading-ampersand nil :url-encode t)))
    (case request-method
      (:get 
       (apply #'drakma:http-request 
      (uri-with-additional-query-part uri param-string-encoded)
      :method request-method
      drakma-args))
      (:post
        (apply #'drakma:http-request
               uri
               :method request-method
               :content param-string-encoded
               drakma-args))

As you can see url-encode is set to t. That was because (I think !) previous versions required encoding and drakma wasn't providing any. Now, ideally tis flag s/b nil. However the issue the becomes the string splitting in dissect-query. This splits on "=" which is also the terminating symbol for the authentication string...
=======================session===================
(drakma::split-string "oauth_signature=oq37d1/qm[....]fIKb778=&include_entities=T&oauth_consumer_key=9[....]cYBg&oauth_token=206[...]Tt5SwRvCJqQWgR3ajEQpk&oauth_signature_method=HMAC-SHA1&oauth_timestamp=1325002586&oauth_nonce=3680613621135035286&oauth_version=1.0" "&")
("oauth_signature=oq37d1/qmFX0YuQUwxsgfIKb778=" "include_entities=T"
[....])

CL-USER> (drakma::split-string "oauth_signature=oq37d1/qmFX0YuQUwxsgfIKb778=" "=")
; compiling (DEFUN HTTP-REQUEST ...)
STYLE-WARNING: redefining DRAKMA:HTTP-REQUEST in DEFUN

("oauth_signature" "oq37d1/qm[...]778")

(I've elided some of the strings...).

I think my proposal resolves the issue. However, it would require a bit more work to get to what I think is the 'right' solution...

At this stage I'd like to get some feedback on whta you consider the right course of action before proceeding to submit a patch

_______________________________________________
drakma-devel mailing list
drakma-devel <at> common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/drakma-devel
Ben Holm | 13 Nov 22:54
Picon

HTTP Put and form-data


Hi all. I started using drakma the other day to do some client work with reviewboard, and I had a little trouble. Reviewboard requires http puts using form-data for updates (http://www.reviewboard.org/docs/manual/1.6/webapi/2.0/overview/). Drakma explicitly disallows this.

I modified request.lisp to let my client do what I needed. Since this is my first time writing a webservice client, I'm not sure if reviewboard or drakma is wrong, or if I misunderstood something. Below is the diff of what I did.

Any thoughts?

Thanks,
-Ben

Index: request.lisp
===================================================================
--- request.lisp (revision 4683)
+++ request.lisp (working copy)
<at> <at> -410,7 +410,7 <at> <at>
     (parameter-error "Don't know how to handle scheme ~S." (uri-scheme uri)))
   (when (and close keep-alive)
     (parameter-error "CLOSE and KEEP-ALIVE must not be both true."))
-  (when (and form-data (not (eq method :post)))
+  (when (and form-data (and (not (eq method :put)) (not (eq method :post))))
     (parameter-error "FORM-DATA makes only sense with POST requests."))
   (when range
     (unless (and (listp range)
<at> <at> -432,7 +432,7 <at> <at>
     (when (and file-parameters-p (not (eq method :post)))
       (parameter-error "Don't know how to handle parameters in ~S, as this is not a POST request."
                        parameters))
-    (when (eq method :post)
+    (when (or (eq method :put) (eq method :post))
       ;; create content body for POST unless it was provided
       (unless content
         ;; mark PARAMETERS argument as used up, so we don't use it

_______________________________________________
drakma-devel mailing list
drakma-devel <at> common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/drakma-devel
Picon

[patch] Resubmit drakma timeout for sbcl.

A long time ago (Dec 2007) there was a patch [1] for drakma to implement
timeouts with sbcl. It was rejected with suggestion to post similar
patch into usocket library.

Totday, I start to learn lisp programing and I need http-request for my
small program. I took drakma for this. But it does not support timeouts on
sbcl. I use debian and it seems sbcl - is the best choice for me.

By googling I found a patch for drakma. I repplaied it for 1.2.3 drakma. It
seems to me, it still forth appling into main line of drakma. I did not
investigate 
what happen with usocket patch and such long time shows - this patch needs 
to be applied anyway :-)

1. http://common-lisp.net/pipermail/drakma-devel/2007-December/000601.html

--
Alexander Litvinov

Attachment (sbcl.patch): text/x-diff, 4694 bytes
_______________________________________________
drakma-devel mailing list
drakma-devel <at> common-lisp.net
http://common-lisp.net/cgi-bin/mailman/listinfo/drakma-devel
Daniel Weinreb | 17 Mar 16:30
Favicon

read-status-line is seeing the HTTP request line!?

Has anybody seen a behavior in which read-status-line
signals a syntax-error, and it turns out that the value
of "line" looks like an HTTP request line rather
than an HTTP status (response) line?  The following
is the stack trace.

 From my old C++ days (may they never return), I
conjectured that some string buffer is being reused
or something like that.; We are running on CCL
(formerly openmcl), and nothing like that is
going on at the CCL level.; I don't know much
about flexi-streams or gray-streams. Anyway
that's just a rather wild guess. Thanks!

<READ-STATUS-LINE at pc +669
     Arguments: (STREAM &amp;OPTIONAL DRAKMA::LOG-STREAM)
       STREAM: <FLEXI-STREAMS:FLEXI-IO-STREAM #x30200938EB3D>
       DRAKMA::LOG-STREAM: NIL
     Local bindings:
       CHUNGA:CURRENT-ERROR-MESSAGE "While reading status line:""
       DRAKMA::LINE: ""POST /stat/ping HTTP/1.1""
       DRAKMA::FIRST-SPACE-POS: 4
       DRAKMA::SECOND-SPACE-POS: 15

Thanks!

- Dan
Edi Weitz | 3 Mar 08:16
Picon
Favicon

Re: http-request error

So, I don't have CLISP to test this, but this looks like an
incompatibility between CLISP and usocket.  If you're using the newest
versions of both, I'd suggest that you ask on the usocket mailing list
and send a full backtrace.

Thanks,
Edi.

On Mon, Feb 28, 2011 at 7:53 PM, Andre Schütz <andre-paul <at> gmx.de> wrote:
> On Mon, 28 Feb 2011 18:06:19 +0100
> Edi Weitz <edi <at> weitz.de> wrote:
>
>> 2011/2/28 "André Schütz" <Andre-Paul <at> gmx.de>:
>>
>> > I loaded the drakma system successfully into my clisp environment and got the following error when I
tried to make a http-request:
>> >
>> > (http-request "http://lisp.org/")
>> >
>> > Fehler: NODELAY in SOCKET-CONNECT is unsupported.
>> >
>> > Any ideas?
>>
>> This is the latest Drakma release or the development version?
>>
>> Also, are you using the newest version of usocket?  Which CLISP
>> version are you on?  Which operating system?
>>
>> Thanks,
>> Edi.
>
> My configuration is as follows:
>
> - clisp: GNU CLISP 2.44.1 (2008-02-23) (built 3476003035)
> - usocket: 0.4.1
> - Operating System: Ubuntu
>
> Andre
>
>
> _______________________________________________
> drakma-devel mailing list
> drakma-devel <at> common-lisp.net
> http://common-lisp.net/cgi-bin/mailman/listinfo/drakma-devel
>
>

Gmane