Picon

ECDSA certificate with curl

Hi,

I tried to use ECDHE-ECDSA-AES128-GCM-SHA256 cipher in the openssl using curl. But curl seems to be not liking ECDSA certificate for server identity when it tried to make https connection. The curl version is latest 7.38.0

Is there any fix for the curl to make it work with ECDSA certificates?

 

Regards,

Ram

 

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Boris Penck | 29 Oct 10:54 2014
Picon

Strange behavior uploading to FTPS with certain cipher suites

Good morning list,

I'm using curl to upload files to a FTPS server with implicit SSL. It works great for tens of thousands of files
but recently I got a little number of damaged files on the target FTPS server (Microsoft FTP Service).

The files in question have additional "garbage" bytes at the end of the file. It looks like binary or a unicode 
potpourri. The files are all UTF-16LE encoded CSV files and looking at the hexdump they look rather normal.
When I delete some random characters from the files, and upload again the files can be ok or still have the 
same issue. The garbage is different in each file but the same for every retry to upload the file.

One of the files causing trouble:

Curl command used for upload:
curl --ftp-create-dirs -vv -k -T $file 'ftps://xxx:xxx <at> xxx.xxx/test1.csv'

What I found out using curl -vv that the connection was using this cipher suite

* SSL connection using ECDHE-RSA-AES256-SHA384

and after messing around with all involved SSL/TLS parameters I was able to upload the file without
any garbage when I force using cipher suite DES-CBC3-SHA. That works for now but it's unknown if
that was the only cause.

Debug output uploading using ECDHE-RSA-AES256-SHA384

The file has the exact same size (7080 bytes) as reported from curl's upload progress, so the additional
bytes are not there before the upload(?)

I don't have access to the FTPS server itself (beside the FTP access) and installed a VSFTPD on another
machine with support for the ECDHE cipher suites and uploaded the file - and there was no garbage 
whatsoever. I've checked that the garbage is indeed on the server and doesn't happen during download.

curl version used:
curl 7.26.0 (x86_64-pc-linux-gnu) libcurl/7.26.0 OpenSSL/1.0.1e zlib/1.2.7 libidn/1.25 libssh2/1.4.2 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap pop3 pop3s rtmp rtsp scp sftp smtp smtps telnet tftp 
Features: Debug GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP 

Most recent package of
Linux proxy1 3.2.0-4-amd64 #1 SMP Debian 3.2.57-3+deb7u1 x86_64 GNU/Linux

I have no idea about Windows FTPS/SSL/TLS integration and for me it looks like incompatibilities between
TLS ciphers on linux and that windows server TLS ciphers but I really have _no_ plan of TLS/SSL ciphers 
and if that's even related to the error.

Does anyone have a little hint for me, what I could look into?
Is it even curl related? 
Is that behavior known elsewhere?

Thanks
Boris
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Nikita Michalko | 28 Oct 14:45 2014

How to find out missing libraries for curl

Hi there,

I currently stuck in configure curl V. 7.38 on the SLES11-SP3 - kernel 3.0.101-0.31; need some assistance - plz help!
Command:
configure --with-libssh2=/usr/lib64/ --with-sftp --enable-proxy --enable-SFTP=YES --with-libidn=/usr/lib64/ --enable-debug --with-gnutls --with-ssl

In config.log I can see:
...
configure:21032: checking if GSS-API support is requested
configure:21163: result: no
configure:21232: checking whether to enable Windows native SSL/TLS (Windows native builds only)
configure:21254: result: no
configure:21270: checking whether to enable iOS/Mac OS X native SSL/TLS
configure:21286: result: no
configure:21328: PKG_CONFIG_LIBDIR will be set to "/usr/lib64/lib/pkgconfig"
configure:21540: checking for CRYPTO_lock in -lcrypto
configure:21562: gcc -o conftest -O2 -Wno-system-headers  -I/usr/lib64/include -I/usr/lib64/include/openssl  -L/usr/lib64/lib conftest.c -lcrypto  -lz -lrt >&5
/usr/lib64/gcc/x86_64-suse-linux/4.3/../../../../x86_64-suse-linux/bin/ld: cannot find -lcrypto
collect2: ld returned 1 exit status

How can I find out which libraries are missing - what exactly is: "lcrypto" ?
And why PKG_CONFIG_LIBDIR will be set to "/usr/lib64/lib/pkgconfig"? How can I change it?

TIA!

Best regards

Nikita Michalko

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Lamont Granquist | 27 Oct 20:34 2014

ca-cert bundle missing Verisign cert, breaking SSL to Amazon

The latest http://curl.haxx.se/ca/cacert.pem drops these cert:

 <at>  <at>  -90,22 +93,6  <at>  <at> 
  70+sB3c4
  -----END CERTIFICATE-----

-Verisign Class 3 Public Primary Certification Authority
-=======================================================
------BEGIN CERTIFICATE-----
-MIICPDCCAaUCEHC65B0Q2Sk0tjjKewPMur8wDQYJKoZIhvcNAQECBQAwXzELMAkGA1UEBhMCVVMx
-FzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMTcwNQYDVQQLEy5DbGFzcyAzIFB1YmxpYyBQcmltYXJ5
-IENlcnRpZmljYXRpb24gQXV0aG9yaXR5MB4XDTk2MDEyOTAwMDAwMFoXDTI4MDgwMTIzNTk1OVow
-XzELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMTcwNQYDVQQLEy5DbGFzcyAz
-IFB1YmxpYyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0aG9yaXR5MIGfMA0GCSqGSIb3DQEBAQUA
-A4GNADCBiQKBgQDJXFme8huKARS0EN8EQNvjV69qRUCPhAwL0TPZ2RHP7gJYHyX3KqhEBarsAx94
-f56TuZoAqiN91qyFomNFx3InzPRMxnVx0jnvT0Lwdd8KkMaOIG+YD/isI19wKTakyYbnsZogy1Ol
-hec9vn2a/iRFM9x2Fe0PonFkTGUugWhFpwIDAQABMA0GCSqGSIb3DQEBAgUAA4GBALtMEivPLCYA
-TxQT3ab7/AoRhIzzKBxnki98tsX63/Dolbwdj2wsqFHMc9ikwFPwTtYmwHYBV4GSXiHx0bH/59Ah
-WM1pF+NEHJwZRDmJXNycAA9WjQKZ7aKQRUzkuxCkPfAyAw7xzvjoyVGM5mKf5p/AfbdynMk2Omuf
-Tqj/ZA1k
------END CERTIFICATE-----
-
[...snip...]
 <at>  <at>  -2610,22 +2397,6  <at>  <at> 
  tkYNbn5XOmeUwssfnHdKZ05phkOTOPu220+DkdRgfks+KzgHVZhepA==
  -----END CERTIFICATE-----

-Verisign Class 3 Public Primary Certification Authority
-=======================================================
------BEGIN CERTIFICATE-----
-MIICPDCCAaUCEDyRMcsf9tAbDpq40ES/Er4wDQYJKoZIhvcNAQEFBQAwXzELMAkGA1UEBhMCVVMx
-FzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMTcwNQYDVQQLEy5DbGFzcyAzIFB1YmxpYyBQcmltYXJ5
-IENlcnRpZmljYXRpb24gQXV0aG9yaXR5MB4XDTk2MDEyOTAwMDAwMFoXDTI4MDgwMjIzNTk1OVow
-XzELMAkGA1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMTcwNQYDVQQLEy5DbGFzcyAz
-IFB1YmxpYyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0aG9yaXR5MIGfMA0GCSqGSIb3DQEBAQUA
-A4GNADCBiQKBgQDJXFme8huKARS0EN8EQNvjV69qRUCPhAwL0TPZ2RHP7gJYHyX3KqhEBarsAx94
-f56TuZoAqiN91qyFomNFx3InzPRMxnVx0jnvT0Lwdd8KkMaOIG+YD/isI19wKTakyYbnsZogy1Ol
-hec9vn2a/iRFM9x2Fe0PonFkTGUugWhFpwIDAQABMA0GCSqGSIb3DQEBBQUAA4GBABByUqkFFBky
-CEHwxWsKzH4PIRnN5GfcX6kb5sroc50i2JhucwNhkcV8sEVAbkSdjbCxlnRhLQ2pRdKkkirWmnWX
-bj9T/UWZYB2oK0z5XqcJ2HUw19JlYD1n1khVdWk/kfVIC0dpImmClr7JyDiGSnoscxlIaU5rfGW/
-D/xwzoiQ
------END CERTIFICATE-----
-
  Microsec e-Szigno Root CA 2009
  ==============================
  -----BEGIN CERTIFICATE-----
 <at>  <at>  -3864,3 +3635,260  <at>  <at> 

I still think i see those certs in the 'upstream' mozilla repo here:

http://hg.mozilla.org/releases/mozilla-release/file/default/security/nss/lib/ckfw/builtins/certdata.txt#l884
http://hg.mozilla.org/releases/mozilla-release/file/default/security/nss/lib/ckfw/builtins/certdata.txt#l17799

I don't see in the history there where anything similar has been removed 
in the mozilla project, either.

If those are being dropped after being scraped, then someone should 
probably be made aware that its a cert at the base of Amazon's SSL certs 
and removing that cert from the ca-bundle breaks 
https://s3.amazonaws.com and https://amazon.com

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Marco Baldassarre | 27 Oct 16:51 2014
Picon

Ship curl command line tool with Mac OS X software

Hi there,
I currently rely in my Mac application on curl command line tool shipped with Mac OS X and installed by
default in /usr/bin/curl.

Altho this approach has worked so far, I have found some inconsistencies between 7.24 and 7.30 versions of
cURL regarding the use of double-quotes marks in the syntax.

As such, I'd like to change my approach and ship a compiled version of cURL with my software, without relying
anymore on the (changing between OS versions) cURL supplied by the Mac OS X platform.

How can I do that? Ideally, I'd like to have a 7.30 version that I can call from
/Applications/MyApp/bin/curl , and that works on Mac OS X from 10.7 to 10.10 without being dependent on
any other library.

Is that even possible?

Thanks for your help
Marco

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html
AC Pan | 27 Oct 11:52 2014
Picon

How to implement long-polling using curl-command-line?

Thanks for your time reading this. I had searched the forum but can't seems to find a way to do this. I also searched extensively in the internet trying to implement using other approaches but did not get it work or too difficult for my level using .


I am trying to implement push notification from a web server using long-polling like https://github.com/panique/php-long-pollingexcept that the client is originated from curl.exe NOT a web browser.

  

Ideally using long-polling approach, the curl.exe client should be able to send a HTTP GET/Post to the Server, STAY CONNECTED for the session until there is new data from the server. 

 

When new data is available, the server return with 1 and some other data from a database, and the connection is closed and the curl client write the data to a file or trigger an external program on the PC.


This way, it will reduce the load and traffic without fixed interval polling from the client. 


Questions:


1. Can curl connect to a php server and have the connection stay with the server until data arrive? How to do it ? i tested Curl.exe, the connection does not stay. May be i am not aware of syntax used. 


 2. If not possible to implement above, how about running an external program that act as a long polling desktop client? what desktop client can be use? 

 

Many thanks again if you guys can offer any advice.

 

Best Regards,
AC Pan
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Oleg Pudeyev | 24 Oct 01:43 2014

Report used SSL library via curl-config

Hi,

PycURL needs to link against the SSL library used by libcurl for locks
in multithreaded environments, which Python is by default. As such
PycURL needs to know what SSL library libcurl is linked against.

curl-config does not directly expose this information. As a result a
number of roundabout ways have been employed to date to figure it out.
Historically `curl-config --libs` was used but some distributions do
not include SSL libraries in this list. There is an option for people
to specify the SSL library explicitly and, most recently, there has
been a request to take SSL library from `curl -V` output[1] - the
problem with this is that PycURL does not require a curl executable to
work. I implemented an alternative solution which loads libcurl.so and
retrieves version information from it[2].

The biggest issue is users want automatic detection of SSL library for
convenience (e.g. installation via pip/easy_install), and automatic
detection is getting increasingly bloated and potentially brittle. For
example, if there are two curl installations on a system and one of
them does not include curl binary but just libcurl, but both include
curl-config, and the libcurl-only installation's curl-config is first
in path, PycURL SSL detection logic would consult curl-config from one
installation and curl from the other.

I would like to request that a facility be added to curl-config to
report which SSL library that particular curl installation is using.

[1] https://github.com/pycurl/pycurl/pull/205
[2] https://github.com/pycurl/pycurl/pull/212

Thanks,
Oleg
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Graeme Clark | 17 Oct 13:01 2014

Dev Versions Missing

http://curl.haxx.se/download.html 

The two Win32 - MSVC dev links 7.18.0 and 7.19.3 are broken. Thought i'd just let someone know. :)

Graeme
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Nick Zitzmann | 17 Oct 07:49 2014

Important note for curl users on OS X Yosemite 10.10

It's time to update this note I wrote for Mavericks,
<http://curl.haxx.se/mail/archive-2013-10/0036.html>, for Yosemite users.

In Yosemite, they switched from version 7.30.0 to 7.37.1. Apple's own Secure Transport engine, not
OpenSSL, is still used for TLS. And there have been a few changes since my last note that affect you if you use
curl to access servers that use TLS:

1. You can now use the --cacert option again, for the purpose of connecting to servers with self-signed
certificates. The catch? It only works with a single PEM- or DER-encoded certificate, and it ignores
additional certificates in the file. This is a known problem in that version.

If this becomes a problem for you, then you can work around it by either:
1a. Using a newer version of curl than the one that comes with Yosemite. curl 7.38.0 supports certificate
bundles when using the Secure Transport engine.
    -or-
1b. Import the certificate bundle into your Keychain, and then discontinue your use of the --cacert option altogether.

I also think that using the --cacert and --insecure options together will result in an error, so don't do
that. :)

2. You can now use the -E/--cert option, for the purpose of authenticating with a TLS host using a client
certificate and private key. When using the option, you can either specify:
2a. The name of the certificate as it appears in your Keychain (the certificate's private key has to be
present in the same Keychain in order for this to work),
    -or-
2b. A path to a PKCS#12-encoded file on a disk, which contains both the certificate and the private key. (If
it's in the present working directory, you need to add a ./ to the start of the path, or curl will assume you
want to search the Keychain.)

Note that the file **must** be in P12 (PKCS#12) format. We can't load client certificates in PEM or DER
format, as well as their private keys, because the API that would be necessary to make that work is
unfortunately private, and I'd rather not have to explain to all of you why I got your curl-utilizing apps
rejected from the App Store.

3. The -2/--sslv2 option will now raise an error if you try to use it. (Previously, the option was ignored.)
Support for SSLv2 was removed from Secure Transport back in OS X 10.8, and it's not coming back. If you need
to access a very old (1995-era) Web site that does not support at least SSLv3 or later, then you'll still
need to build your own curl and use OpenSSL instead.

4. The --ssl-allow-beast option will now work, but I don't recommend using it unless you **really** know
what you're doing. By default, curl will try to work around the BEAST problem when connecting to a site that
uses CBC over TLS 1.0.

Nick Zitzmann
<http://www.chronosnet.com/>

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Bruno Thomsen | 15 Oct 12:48 2014
Picon

[PATCH] mk-ca-bundle: added SHA-384 signature algorithm

Certificates based on SHA-1 are being phased out[1].
So we should expect a rise in certificates based on SHA-2.
Adding SHA-384 as a valid signature algorithm.

[1] https://blog.mozilla.org/security/2014/09/23/phasing-out-certificates-with-sha-1-based-signature-algorithms/

Signed-off-by: Bruno Thomsen <bth <at> kamstrup.dk>
---
 docs/mk-ca-bundle.1 | 2 +-
 lib/mk-ca-bundle.pl | 3 ++-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/docs/mk-ca-bundle.1 b/docs/mk-ca-bundle.1
index aa38612..7d38dba 100644
--- a/docs/mk-ca-bundle.1
+++ b/docs/mk-ca-bundle.1
 <at>  <at>  -87,7 +87,7  <at>  <at>  each certificate and output when run in plain text mode.

 Valid algorithms are:
 .RS
-ALL, NONE, MD5 (default), SHA1, SHA256, SHA512
+ALL, NONE, MD5 (default), SHA1, SHA256, SHA384, SHA512
 .RE
 .IP -u
 unlink (remove) certdata.txt after processing
diff --git a/lib/mk-ca-bundle.pl b/lib/mk-ca-bundle.pl
index 51af5c9..4278e82 100755
--- a/lib/mk-ca-bundle.pl
+++ b/lib/mk-ca-bundle.pl
 <at>  <at>  -56,7 +56,7  <at>  <at>  $opt_d = 'release';
 # If the OpenSSL commandline is not in search path you can configure it here!
 my $openssl = 'openssl';

-my $version = '1.23';
+my $version = '1.24';

 $opt_w = 76; # default base64 encoded lines length

 <at>  <at>  -97,6 +97,7  <at>  <at>  my  <at> valid_signature_algorithms = (
   "MD5",
   "SHA1",
   "SHA256",
+  "SHA384",
   "SHA512"
 );

--

-- 
1.9.1

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html
Al Grant | 11 Oct 23:48 2014
Picon

Newbie help : logging into website with cookies

Hi All,

I would really appreicate any help on this. I have been trying to
login to a website and POST data to a second page for about 3 days now
without success.

I can acheive a login by copying the curl commands from firefox, but
if I try to script it, I fail. In essence when I try to script it I am
removing the -H "Cookie:...." for -b

Here is what I get from FF:

curl "https://www.anpronline.net/j_spring_security_check" -H "Host:
www.anpronline.net" -H "User-Agent: Mozilla/5.0 (Windows NT 6.1;
WOW64; rv:32.0) Gecko/20100101 Firefox/32.0" -H "Accept:
text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8" -H
"Accept-Language: en-GB,en;q=0.5" -H "Accept-Encoding: gzip, deflate"
-H "Referer: https://www.anpronline.net/index.html" -H "Cookie:
__utma=86946376.583746695.1412828339.1413009922.1413057495.9;
__utmz=86946376.1412828339.1.1.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not"%"20provided);
JSESSIONID=113idx1ipor6y10oul7yy6uaoi;
AWSELB=A7A737371AF52D0B544DAF902D9C2A0C5FBFC37325B3CB707E1EFDCF041AE1E91355018DDFD9065F26A21E422A304806A548114E34518568CA193FB0649451B57AE20FD01E;
__utmc=86946376; __utmb=86946376.10.10.1413057495; __utmt=1" -H
"Connection: keep-alive" --data
"j_username=al"%"40test.co.nz&j_password=SECRET"

curl "https://www.anpronline.net/blacklists/bl.html" -H "Host:
www.anpronline.net" -H "User-Agent: Mozilla/5.0 (Windows NT 6.1;
WOW64; rv:32.0) Gecko/20100101 Firefox/32.0" -H "Accept:
text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8" -H
"Accept-Language: en-GB,en;q=0.5" -H "Accept-Encoding: gzip, deflate"
-H "Referer: https://www.anpronline.net/blacklists.html" -H "Cookie:
__utma=86946376.583746695.1412828339.1413009922.1413057495.9;
__utmz=86946376.1412828339.1.1.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not"%"20provided);
JSESSIONID=kyr8s0yhfj408gbfauj4yroh;
AWSELB=A7A737371AF52D0B544DAF902D9C2A0C5FBFC37325B3CB707E1EFDCF041AE1E91355018DDFD9065F26A21E422A304806A548114E34518568CA193FB0649451B57AE20FD01E;
__utmc=86946376; __utmb=86946376.14.10.1413057495; __utmt=1" -H
"Connection: keep-alive" --data "name=test&siteId=0"

And to script it:

#!/bin/bash
#new
curl --cookie-jar /tmp/anpronline.cookie "https://anpronline.net"

curl "https://www.anpronline.net/j_spring_security_check" -H "Host:
www.anpronline.net" -H "User-Agent: Mozilla/5.0 (Windows NT 6.1;
WOW64; rv:32.0) Gecko/20100101 Firefox/32.0" -H "Accept:
text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8" -H
"Accept-Language: en-GB,en;q=0.5" -H "Accept-Encoding: gzip, deflate"
-H "Referer: https://www.anpronline.net/index.html" "Connection:
keep-alive" -b /tmp/anpronline.cookie --data
"j_username=al"%"40test.co.nz&j_password=SECRET"

curl "https://www.anpronline.net/blacklists/bl.html" -H "Host:
www.anpronline.net" -H "User-Agent: Mozilla/5.0 (Windows NT 6.1;
WOW64; rv:32.0) Gecko/20100101 Firefox/32.0" -H "Accept:
text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8" -H
"Accept-Language: en-GB,en;q=0.5" -H "Accept-Encoding: gzip, deflate"
-H "Referer: https://www.anpronline.net/blacklists.html" -b
/tmp/anpronline.cookie -H "Connection: keep-alive" --data
"name=test&siteId=0"

Could someone please help?

Many thanks,

--

-- 
"Beat it punk!"
- Clint Eastwood
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html

Gmane