Brad Fitzpatrick | 12 Sep 21:07 2014

spin when http/2 connection is closed prematurely

I'm using curl with nghttp2 for integration tests with development of Go's http2 support. I'm building curl like: https://github.com/bradfitz/http2/blob/master/Dockerfile (curl curl-7.38.0)

There seems to be a bug with curl's use of nghttp2.

If the server closes the connection prematurely (after NPN/ALPN negotiation but before sending any http/2 frames), curl --http2 -v spins forever with:

* nghttp2_session_mem_recv() returns 0
* http2_recv: 16384 bytes buffer
* nread=0
* nghttp2_session_mem_recv() returns 0
* http2_recv: 16384 bytes buffer
* nread=0
* nghttp2_session_mem_recv() returns 0
* http2_recv: 16384 bytes buffer
* nread=0
* nghttp2_session_mem_recv() returns 0
* http2_recv: 16384 bytes buffer
* nread=0
* nghttp2_session_mem_recv() returns 0
* http2_recv: 16384 bytes buffer
* nread=0
* nghttp2_session_mem_recv() returns 0
* http2_recv: 16384 bytes buffer
* nread=0
* nghttp2_session_mem_recv() returns 0
* http2_recv: 16384 bytes buffer
* nread=0
* nghttp2_session_mem_recv() returns 0
* http2_recv: 16384 bytes buffer
* nread=0
* nghttp2_session_mem_recv() returns 0
* http2_recv: 16384 bytes buffer
* nread=0
* nghttp2_session_mem_recv() returns 0
* http2_recv: 16384 bytes buffer
* nread=0
* nghttp2_session_mem_recv() returns 0
* http2_recv: 16384 bytes buffer
* nread=0
* nghttp2_session_mem_recv() returns 0
* http2_recv: 16384 bytes buffer
* nread=0

etc

I'll work around this with a wrapper, but seems like something curl should fix.

Let me know if you'd prefer that I file a bug.
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Peterholm | 10 Sep 14:25 2014
Picon
Picon

aborted

updating from 7.19 to 7.27 and having a few users now getting : recv failure. connection was aborted. any ideas ?


Sendt fra Samsung mobil
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Daniel Stenberg | 10 Sep 08:40 2014
Picon

[SECURITY ADVISORY] cookie leak for TLDs

                          libcurl cookie leak for TLDs
                          ============================

Project cURL Security Advisory, September 10th 2014
http://curl.haxx.se/docs/security.html

1. VULNERABILITY

   libcurl wrongly allows cookies to be set for Top Level Domains (TLDs), thus
   making them apply broader than cookies are allowed. This can allow arbitrary
   sites to set cookies that then would get sent to a different and unrelated
   site or domain.

2. INFO

   Cookie parsing and use is opt-in by applications and is not enabled by
   default.

   libcurl's cookie parser has no Public Suffix awareness, so apart from
   rejecting TLDs from being allowed it might still allow cookies for domains
   that are otherwise widely rejected by ordinary browsers. See
   https://publicsuffix.org/ for details.

   The Common Vulnerabilities and Exposures (CVE) project has assigned the name
   CVE-2014-3620 to this issue.

3. AFFECTED VERSIONS

   Affected versions: from libcurl 7.31.0 to and including 7.37.1
   Not affected versions: libcurl < 7.31.0 and libcurl >= 7.38.0

   libcurl is used by many applications, but not always advertised as such!

4. THE SOLUTION

   libcurl 7.38.0 doesn't accept cookies set for just a TLD. Note that it does
   not add any public suffix awareness apart from that.

   A patch for this problem is available at:

     http://curl.haxx.se/CVE-2014-3620.patch

5. RECOMMENDATIONS

   We suggest you take one of the following actions immediately, in order of
   preference:

   A - Upgrade to curl and libcurl 7.38.0

   B - Apply the patch and rebuild libcurl

   C - Avoid using cookies in your application

6. TIME LINE

   It was reported to the curl project on August 15th 2014. We contacted
   distros <at> openwall on September 1st.

   libcurl 7.38.0 was released on September 10th 2014, coordinated with the
   publication of this advisory.

7. CREDITS

   Reported by Tim Ruehsen. Patch written by Daniel Stenberg.

   Thanks a lot!

--

-- 

  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html
j3v | 2 Sep 21:29 2014
Picon

Upload to Flickr using plain cURL

Hello.
I am a noob trying to upload photos to Flickr using plain cURL.
I have been looking at their API (https://www.flickr.com/services/api/auth.howto.desktop.html) and I have read a couple of forum topics on the subject here and here.
Could someone please put me on the right track?
Thanks in advance. :-)
Cheers
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Stephen Röttger | 2 Sep 14:32 2014
Picon

Disable accepting MD5 certificates

Hi,


I noticed that the curl command line tool on Linux is accepting MD5 certificates and couldn't figure out how to disable this behavior. For my test setup, I created a ca certificate signed with sha256WithRSAEncryption and a server cert with md5WithRSAEncryption and ran 'curl https://mydomain.com/ --cacert ca.crt' which would happily connect to the server.
Even though no CA is issuing MD5 certs anymore (I hope), this still poses a security risk if an attacker is in possession of an expired rogue CA certificate similar to [0]. The expiry check can often be bypassed since many clients synchronize their time with an external source without any authentication.
Is there a way to disable accepting MD5 certificates? I assume the same issue applies to the libcurl easy interface as well.


Thanks,
Stephen
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html
barry Chung | 29 Aug 02:16 2014
Picon

RE: Difference in curl performance between RHEL6 and RHEL7

On Tuesday, August 12, 2014 12:29:55 Alastair Scobie wrote: 
>> On 12/08/14 12:18, Kamil Dudka wrote: 
>> > On Tuesday, August 12, 2014 10:05:37 Alastair Scobie wrote: 
>> >> Whilst porting an in-house application, which uses libcurl, from RHEL6 
>> >> to RHEL7 we noticed that the time taken by libcurl to fetch a page had 
>> >> substantially increased. Even an attempt at fetching a non-existent page 
>> >> has increased from 10-20ms to around 150ms. This with just plain HTTP. 
>> > 
>> > What exactly do you mean by a non-existent page? 
>> 
>> A page for which you'll receive a 404. Ie. HTTP server address is 
>> correct, but actual page requested doesn't exist. 
>> 
>> >> Our first thought was that the API has changed in some subtle way and 
>> >> the fault must be in our code, but then we thought to try measuring the 
>> >> performance of the curl tool itself. To our surprise, we found the same 
>> >> performance disparity. 
>> >> 
>> >> Interestingly... 
>> >> 
>> >> * Adding the DNS address of the HTTP server to /etc/hosts (and 
>> >> specifying files,dns in /etc/nsswitch.conf) does not improve things 
>> > 
>> > Did you try to measure the time consumed by 'getent ahosts'? 
>> 
>> Small number of ms for both RHEL6 and RHEl7. 
>> 
>> >> * but, performance when using an IP address in the URL is fine (~ 
>> >> 10-20ms) for both RHEL6 and RHEL7 
>> > 
>> > I was not able to reproduce this behavior. Is it reproducible locally? 
>> 
>> Yip, on several machines. I can ship you an strace if you like? 
>I would prefer a self-contained program that I can use to repeat the bug. 
>> (Daniel's suggestion of disabling the threaded resolver has fixed the 
>> issue for us). 
>> 
>> Cheers, Alastair 
>The problem is that either you will not receive security updates for (lib)curl 
>any more, or the threaded resolver will be re-enabled on the next update. 
>Kamil 

We are also hitting this while porting our software to RHEL7. Like Alastair
mentioned, using IP address does not have the issue. wget also works fine.

Kamil, you should be reproduce this by 
1) bring up a RHEL7 minimal installation
2) install httpd or lighttpd
3) sudo tail -f /var/log/lighttpd/access.log
4a) while [ 1 ]; do curl http://localhost/badpage; done
4b) while [ 1 ]; do curl http://127.0.0.1/badpage; done

There will be huge difference in the access count per second between 4a) and 4b).

BTW, I uninstalled 7.29.0-19.el7 and installed 7.37.1-3.0.cf.rhel7 from
http://curl.haxx.se/download.html but the problem still seems to exist.

Thanks,
Barry
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Anders Palm | 25 Aug 20:04 2014

Problems linking ssl when cross-compiling curl

Hello curl-users!

I'm trying to cross-compile curl 7.37.0 with openssl for an arm target.

Openssl is correctly build for the target with a custom prefix. It is 
installed in its collected usr/local/ssl fashion.

./configure --host=arm-none-linux --with-ssl=${ARM_TOOLS_DIR}/usr/local/ssl

Correctly identifies where openssl is:

checking whether to enable Windows native SSL/TLS (Windows native builds 
only)... no
checking whether to enable iOS/Mac OS X native SSL/TLS... no
configure: PKG_CONFIG_LIBDIR will be set to 
"/home/palm/projects/build_tools/build/sys-roots/install-arm-dir/usr/local/ssl/lib/pkgconfig"
checking for arm-none-linux-pkg-config... no
checking for pkg-config... /usr/bin/pkg-config
checking for openssl options with pkg-config... found
configure: pkg-config: SSL_LIBS: "-lssl -lcrypto -ldl  "
configure: pkg-config: SSL_LDFLAGS: 
"-L/home/palm/projects/build_tools/build/sys-roots/install-arm-dir/usr/local/ssl/lib 
"
configure: pkg-config: SSL_CPPFLAGS: 
"-I/home/palm/projects/build_tools/build/sys-roots/install-arm-dir/usr/local/ssl/include 
"
checking for CRYPTO_lock in -lcrypto... yes
checking for SSL_connect in -lssl... yes
checking openssl/x509.h usability... yes
checking openssl/x509.h presence... yes
checking for openssl/x509.h... yes
checking openssl/rsa.h usability... yes
checking openssl/rsa.h presence... yes
checking for openssl/rsa.h... yes
checking openssl/crypto.h usability... yes
checking openssl/crypto.h presence... yes
checking for openssl/crypto.h... yes
checking openssl/pem.h usability... yes
checking openssl/pem.h presence... yes
checking for openssl/pem.h... yes
checking openssl/ssl.h usability... yes
checking openssl/ssl.h presence... yes
checking for openssl/ssl.h... yes
checking openssl/err.h usability... yes
checking openssl/err.h presence... yes
checking for openssl/err.h... yes
checking openssl/pkcs12.h usability... yes
checking openssl/pkcs12.h presence... yes
checking for openssl/pkcs12.h... yes
checking for ENGINE_init... yes
checking openssl/engine.h usability... yes
checking openssl/engine.h presence... yes
checking for openssl/engine.h... yes
checking for ENGINE_load_builtin_engines... yes
checking for RAND_status... yes
checking for RAND_screen... no
checking for RAND_egd... yes
checking for ENGINE_cleanup... yes
checking for CRYPTO_cleanup_all_ex_data... yes
checking for SSL_get_shutdown... yes
checking for SSLv2_client_method... yes
checking for SSL_CTX_set_next_proto_select_cb... no
checking for SSL_CTX_set_alpn_protos... no
checking for SSL_CTX_set_alpn_select_cb... no
checking for yaSSL using OpenSSL compatibility mode... no
checking for OpenSSL headers version... 0.9.8 - 0x009080bfL
checking for OpenSSL library version... 0.9.8
checking for OpenSSL headers and library versions matching... yes

But when running make, i get the following errors:

/bin/bash ../libtool  --tag=CC   --mode=link arm-none-linux-gnueabi-gcc  
-march=armv4t -O2 -Wno-system-headers 
-L/home/palm/projects/build_tools/build/sys-roots/install-arm-dir/usr/local/ssl/lib 
-o curl curl-tool_binmode.o curl-tool_bname.o curl-tool_cb_dbg.o 
curl-tool_cb_hdr.o curl-tool_cb_prg.o curl-tool_cb_rea.o 
curl-tool_cb_see.o curl-tool_cb_wrt.o curl-tool_cfgable.o 
curl-tool_convert.o curl-tool_dirhie.o curl-tool_doswin.o 
curl-tool_easysrc.o curl-tool_formparse.o curl-tool_getparam.o 
curl-tool_getpass.o curl-tool_help.o curl-tool_helpers.o 
curl-tool_homedir.o curl-tool_hugehelp.o curl-tool_libinfo.o 
curl-tool_main.o curl-tool_metalink.o curl-tool_mfiles.o 
curl-tool_msgs.o curl-tool_operate.o curl-tool_operhlp.o 
curl-tool_panykey.o curl-tool_paramhlp.o curl-tool_parsecfg.o 
curl-tool_setopt.o curl-tool_sleep.o curl-tool_urlglob.o 
curl-tool_util.o curl-tool_vms.o curl-tool_writeenv.o 
curl-tool_writeout.o curl-tool_xattr.o ../lib/curl-strtoofft.o 
../lib/curl-strdup.o ../lib/curl-rawstr.o ../lib/curl-nonblock.o 
../lib/curl-warnless.o  ../lib/libcurl.la   -lrt
libtool: link: arm-none-linux-gnueabi-gcc -march=armv4t -O2 
-Wno-system-headers -o .libs/curl curl-tool_binmode.o curl-tool_bname.o 
curl-tool_cb_dbg.o curl-tool_cb_hdr.o curl-tool_cb_prg.o 
curl-tool_cb_rea.o curl-tool_cb_see.o curl-tool_cb_wrt.o 
curl-tool_cfgable.o curl-tool_convert.o curl-tool_dirhie.o 
curl-tool_doswin.o curl-tool_easysrc.o curl-tool_formparse.o 
curl-tool_getparam.o curl-tool_getpass.o curl-tool_help.o 
curl-tool_helpers.o curl-tool_homedir.o curl-tool_hugehelp.o 
curl-tool_libinfo.o curl-tool_main.o curl-tool_metalink.o 
curl-tool_mfiles.o curl-tool_msgs.o curl-tool_operate.o 
curl-tool_operhlp.o curl-tool_panykey.o curl-tool_paramhlp.o 
curl-tool_parsecfg.o curl-tool_setopt.o curl-tool_sleep.o 
curl-tool_urlglob.o curl-tool_util.o curl-tool_vms.o 
curl-tool_writeenv.o curl-tool_writeout.o curl-tool_xattr.o 
../lib/curl-strtoofft.o ../lib/curl-strdup.o ../lib/curl-rawstr.o 
../lib/curl-nonblock.o ../lib/curl-warnless.o 
-L/home/palm/projects/build_tools/build/sys-roots/install-arm-dir/usr/local/ssl/lib 
../lib/.libs/libcurl.so
-lrt
/home/palm/projects/build_tools/build/sys-roots/install-arm-dir/packages/arm_toolchain/bin/../lib/gcc/arm-none-linux-gnueabi/4.3.3/../../../../arm-none-linux-gnueabi/bin/ld: 
warning: libssl.so.0.9.8, needed by ../lib/.libs/libcurl.so, not found 
(try using -rpath or
-rpath-link)
/home/palm/projects/build_tools/build/sys-roots/install-arm-dir/packages/arm_toolchain/bin/../lib/gcc/arm-none-linux-gnueabi/4.3.3/../../../../arm-none-linux-gnueabi/bin/ld: 
warning: libcrypto.so.0.9.8, needed by ../lib/.libs/libcurl.so, not 
found (try using -rpath or -rpath-link)
../lib/.libs/libcurl.so: undefined reference to `SSL_connect'
../lib/.libs/libcurl.so: undefined reference to `X509_check_issued'
../lib/.libs/libcurl.so: undefined reference to `BIO_free'
../lib/.libs/libcurl.so: undefined reference to `BIO_s_mem'
../lib/.libs/libcurl.so: undefined reference to `UI_method_get_reader'
../lib/.libs/libcurl.so: undefined reference to `SSL_get_session'
../lib/.libs/libcurl.so: undefined reference to `UI_get_string_type'
../lib/.libs/libcurl.so: undefined reference to `sk_pop_free'
../lib/.libs/libcurl.so: undefined reference to `BIO_ctrl'
....

I've tried searching the mailing list for similar problems, but not 
found anyone with this exact issue. Any help is very much appreciated

// Anders Palm
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Bisera Milosheska | 24 Aug 09:39 2014
Picon

../lib/.libs/libcurl.so: undefined reference to `SSLv2_client_method'


I have openssl-1.0.1i and I am getting this error when I try to install curl-7.38.0-DEV:

	../lib/.libs/libcurl.so: undefined reference to `SSLv2_client_method’

I have seen some solutions that I can make changes to the ssluse.c file, but there is no such file in my
curl/lib/ directory. Do you have any suggestions on how to solve this?

Thank you in advance.

Best regards,
Bisera
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Bisera Milosheska | 22 Aug 16:36 2014
Picon

Curl and HTTP/2

I have the latest version of curl, 7.37.1. It is supposed to support http2, but when I try to do a request to a nghttpd server(which is part of the nghttp2 implementation) that supports http2, the server answers that the client did not advertise HTTP/2 protocol. This is my command:

curl --http2 -k -n100 -c10 -m10 'https://10.0.0.10:8080/index.html'

The server is working fine, it can receive HTTP/2.0 requests from the client available in the nghttp2 implementation. Is my command not correct or is there some problem with the curl support of HTTP/2?
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Jenkins, Peter (SLSA | 20 Aug 06:28 2014
Picon
Picon

How to configure curl with libssh2

I want to use curl to download files from a sftp server and if I am reading the documentation correctly, curl
needs libssh2.

The curl version installed in OS X 10.9.4:
    curl 7.30.0 (x86_64-apple-darwin13.0) libcurl/7.30.0 SecureTransport zlib/1.2.5
    Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smtp smtps telnet tftp 
    Features: AsynchDNS GSS-Negotiate IPv6 Largefile NTLM NTLM_WB SSL libz 

As there is no libbssh2, I installed libssh2-1.4.3 and now I have new folders and various libssh2 files in /usr/local.

I guessed that I would have to install a new version of curl that includes libbssh2.
From the documentation I expect this curl to install in /usr/local, which would seem to match well with my
libssh2 install.

I ran the command ./configure --with-darwinssl --with-libssh2
and got this error:
    configure: error: libSSH2 libs and/or directories were not found where specified

I didn’t specify where to find "libSSH2 libs and/or directories”, assuming that /usr/local would be
the default.
Neither the output nor the config.log indicate where configure was looking.

How do I fix this?

Am I on the right track anyway?

Peter Jenkins
State Library of South Australia

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Gisle Vanem | 19 Aug 20:51 2014
Picon

Making curl.exe UNICODE aware

The libcurl.dll works fine when compiled with -DUNICODE,
but the curl.exe tool doesn't work at all. I was getting 
'curl (27): No memory' on all URLs. The easy fix for me was to 
explicitly call the ASCII version for the below functions. 

Patch:

--- orig/src/tool_homedir.c        2014-04-18 19:21:58 +0000
+++ src/tool_homedir.c    2014-08-19 19:55:07 +0000
 <at>  <at>  -39,14 +39,14  <at>  <at> 
   /* Don't use getenv(); it doesn't find variable added after program was
    * started. Don't accept truncated results (i.e. rc >= sizeof(buf1)).  */

-  rc = GetEnvironmentVariable(variable, buf1, sizeof(buf1));
+  rc = GetEnvironmentVariableA(variable, buf1, sizeof(buf1));
   if(rc > 0 && rc < sizeof(buf1)) {
     env = buf1;
     variable = buf1;
   }
   if(do_expand && strchr(variable,'%')) {
     /* buf2 == variable if not expanded */
-    rc = ExpandEnvironmentStrings (variable, buf2, sizeof(buf2));
+    rc = ExpandEnvironmentStringsA (variable, buf2, sizeof(buf2));
     if(rc > 0 && rc < sizeof(buf2) &&
        !strchr(buf2,'%'))    /* no vars still unexpanded */
       env = buf2;
--- orig/src/tool_parsecfg.c       2014-04-18 19:21:58 +0000
+++ src/tool_parsecfg.c   2014-08-19 19:54:44 +0000
 <at>  <at>  -79,7 +79,7  <at>  <at> 
            * already declared via inclusions done in setup header file.
            * We assume that we are using the ASCII version here.
            */
-          int n = GetModuleFileName(0, filebuffer, sizeof(filebuffer));
+          int n = GetModuleFileNameA(0, filebuffer, sizeof(filebuffer));
           if(n > 0 && n < (int)sizeof(filebuffer)) {
             /* We got a valid filename - get the directory part */
             char *lastdirchar = strrchr(filebuffer, '\\');

--------------

I have no idea where and why I got this "'curl (27): No memory" before
the above patch. But looks like some failure in tool_parsecfg.c. Although
CURLE_OUT_OF_MEMORY is returned elsewhere.

--gv
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html

Gmane