isshed | 25 Jul 09:22 2016
Picon

Crash in curl library while processing HTTP HEAD response

Hi Experts,

I am new to the curl library. I am facing a crash in the curl library
while processing HTTP HEAD Request. Crash dump stack is showing like
as follows. could anybody knows if this is fixed or not?

#############################
#0  0x40a91ab0 in ?? ()
#1  0x00463b48 in Curl_client_write (conn=0x15ea3d0, type=1, ptr=0x1
<Address 0x1 out of bounds>, len=0) at
/Code/ThirdParty/curl/lib/sendf.c:494
#2  0x00463b48 in Curl_client_write (conn=0x15ea3d0, type=1,
ptr=0x15f53d0 "404 - Error: Page Not Found.\r\n", len=30) at
/Code/ThirdParty/curl/lib/sendf.c:494
#3  0x00466704 in readwrite_data (done=<value optimized out>,
didwhat=<value optimized out>, k=<value optimized out>, conn=<value
optimized out>, data=<value optimized out>)
    at /Code/ThirdParty/curl/lib/transfer.c:613
#4  Curl_readwrite (conn=0x15ea3d0, done=0x15ea490 "") at
/Code/ThirdParty/curl/lib/transfer.c:1620
#5  0x00466ee4 in Transfer (conn=<value optimized out>) at
/Code/ThirdParty/curl/lib/transfer.c:1876
#6  Curl_perform (data=0x15d4670) at /Code/ThirdParty/curl/lib/transfer.c:2501
#############################

Thanks,
Isshed
-------------------------------------------------------------------
List admin: https://cool.haxx.se/list/listinfo/curl-users
FAQ:        https://curl.haxx.se/docs/faq.html
(Continue reading)

isshed | 22 Jul 15:24 2016
Picon

Crash in curl library while processing HTTP HEAD response

Hi Experts,

I am new to the curl library. I am facing a crash in the curl library
while processing HTTP HEAD Request. Crash dump stack is showing like
as follows.

#############################
#0  0x40a91ab0 in ?? ()
#1  0x00463b48 in Curl_client_write (conn=0x15ea3d0, type=1, ptr=0x1
<Address 0x1 out of bounds>, len=0) at
/Code/ThirdParty/curl/lib/sendf.c:494
#2  0x00463b48 in Curl_client_write (conn=0x15ea3d0, type=1,
ptr=0x15f53d0 "404 - Error: Page Not Found.\r\n", len=30) at
/Code/ThirdParty/curl/lib/sendf.c:494
#3  0x00466704 in readwrite_data (done=<value optimized out>,
didwhat=<value optimized out>, k=<value optimized out>, conn=<value
optimized out>, data=<value optimized out>)
    at /Code/ThirdParty/curl/lib/transfer.c:613
#4  Curl_readwrite (conn=0x15ea3d0, done=0x15ea490 "") at
/Code/ThirdParty/curl/lib/transfer.c:1620
#5  0x00466ee4 in Transfer (conn=<value optimized out>) at
/Code/ThirdParty/curl/lib/transfer.c:1876
#6  Curl_perform (data=0x15d4670) at /Code/ThirdParty/curl/lib/transfer.c:2501
#############################

Looks like curl has received "404 Error:Page Not Found" response for
HTTP HEAD request from server. I have made few code changes in the
readwrite_data()
function in file transfer.c. The code changes are under "ISSHED start"
and "ISSHED stop" block. Here k->badheader is being set as
(Continue reading)

Spork Schivago | 17 Jul 00:08 2016
Picon

Re: How to permanently disable ciphers in curl command line.

Thank you for the response, but I should have been more clear.

Is there away to disable it system wide?   For example, is there a config file somewheres that curl reads from or something where I can specifically say --ciphers 'ALL:!SHA!:!MD5:!aNULL'  so they're never used by default?   Or would I have to download the source file and custom build curl and it's library?

Thank you for taking the time to answer my question.   It's much appreciated.

Ken

On Sat, Jul 16, 2016 at 12:55 PM, Anonymous Remailer (austria) <mixmaster <at> remailer.privacy.at> wrote:

curl --ciphers 'ALL:!SHA1:!MD5:!aNULL' https://www.howsmyssl.com/a/check

-------------------------------------------------------------------
List admin: https://cool.haxx.se/list/listinfo/curl-users
FAQ:        https://curl.haxx.se/docs/faq.html
Etiquette:  https://curl.haxx.se/mail/etiquette.html
Spork Schivago | 15 Jul 20:11 2016
Picon

How to permanently disable ciphers in curl command line.

Hello,

I'm sorry if this is a dumb question.  I'm running a Virtual Private Server (VPS) that has CentOS 6.8 Final on it.  curl is version 7.19.7 (x86_64-redhat-linux-gnu).   I run this command to get an idea of what ciphers are installed:

curl https://www.howsmyssl.com/a/check

And I see this response:

{"given_cipher_suites":["TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256","TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256","TLS_DHE_RSA_WITH_AES_128_GCM_SHA256","TLS_RSA_WITH_AES_128_GCM_SHA256","TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA","TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA","TLS_DHE_RSA_WITH_AES_256_CBC_SHA","TLS_DHE_RSA_WITH_AES_256_CBC_SHA256","TLS_DHE_DSS_WITH_AES_256_CBC_SHA","TLS_RSA_WITH_AES_256_CBC_SHA","TLS_RSA_WITH_AES_256_CBC_SHA256","TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA","TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA","TLS_DHE_RSA_WITH_AES_128_CBC_SHA","TLS_DHE_RSA_WITH_AES_128_CBC_SHA256","TLS_DHE_DSS_WITH_AES_128_CBC_SHA","TLS_RSA_WITH_AES_128_CBC_SHA","TLS_RSA_WITH_AES_128_CBC_SHA256","TLS_RSA_WITH_RC4_128_SHA","TLS_RSA_WITH_RC4_128_MD5","TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA","TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA","TLS_RSA_WITH_3DES_EDE_CBC_SHA"],"ephemeral_keys_supported":true,"session_ticket_supported":false,"tls_compression_supported":false,"unknown_cipher_suite_supported":false,"beast_vuln":false,"able_to_detect_n_minus_one_splitting":false,"insecure_cipher_suites":{"TLS_RSA_WITH_RC4_128_MD5":["uses RC4 which has insecure biases in its output"],"TLS_RSA_WITH_RC4_128_SHA":["uses RC4 which has insecure biases in its output"]},"tls_version":"TLS 1.2","rating":"Bad"}

It's those last two that really worry me.   The TLS_RSA_WITH_RC4_128_MD5 cipher and the TLS_RSA_WITH_RC4_128_SHA cipher.   Is there away to disable them system wide?   I don't know a lot about curl and I've tried searching the net but couldn't really find much on it.   Any suggestions are welcomed.

I'd like to add that I'm new to operating a server so please go easy on me!

Thanks!

Spork
-------------------------------------------------------------------
List admin: https://cool.haxx.se/list/listinfo/curl-users
FAQ:        https://curl.haxx.se/docs/faq.html
Etiquette:  https://curl.haxx.se/mail/etiquette.html
el kalin | 14 Jul 19:47 2016
Picon

ssh custom port



hi all...    

i have been asked to send files to a remote machine via sftp using a non standard ssh port.

the sftp tool has the option -oPort that allows for a non standard port to be used for the ssh hookup.
i've used curl many times to scrip sftp transfers but can't figure out how to tell curl to use a port other than 22 for an sftp transfer...

i keep getting:

* About to connect() to remote.host.com port 10022 (#0)
*   Trying 12.34.56.78... connected
* Connected to remote.host.com (12.34.56.78) port 10022 (#0)
* Failure establishing ssh session
* Closing connection #0

curl: (2) Failure establishing ssh session

i tried pretty much any option including sftp://remote.host.com:10022...

i know it works with sftp -oPort=10022 but i can't script sftp. what's the equivalent of that in curl?  

any ideas welcomed... 

thanks.. 

-------------------------------------------------------------------
List admin: https://cool.haxx.se/list/listinfo/curl-users
FAQ:        https://curl.haxx.se/docs/faq.html
Etiquette:  https://curl.haxx.se/mail/etiquette.html
Jan Birk | 9 Jul 19:33 2016
Picon

Wrong total_count ?

Hi,

I am not sure I understand the -w option completely :

Using

curl -w  <at> curl-format.txt .....

and curl-format.txt is:
---------------
  cat curl-format.txt
     time_namelookup:  %{time_namelookup}\n
        time_connect:  %{time_connect}\n
     time_appconnect:  %{time_appconnect}\n
    time_pretransfer:  %{time_pretransfer}\n
       time_redirect:  %{time_redirect}\n
  time_starttransfer:  %{time_starttransfer}\n
  -------------------- ---------------------\n
    time_total (sec):  %{time_total}\n

The %time_total don’t add up? It gives:

     time_namelookup:  0,125
        time_connect:  0,127
     time_appconnect:  0,000
    time_pretransfer:  0,127
       time_redirect:  0,000
  time_starttransfer:  0,160
  -------------------- ---------------------
    time_total (sec):  0,160

Isn't time_total supposed to be the sum of all variables or are some of 
the variable not included or included?

Best and thanks

/Jan

--

-- 
Med venlig hilsen

Jan Birk
Høvedstensvej 13
2650 Hvidovre
Tlf: 0045 41741615

-------------------------------------------------------------------
List admin: https://cool.haxx.se/list/listinfo/curl-users
FAQ:        https://curl.haxx.se/docs/faq.html
Etiquette:  https://curl.haxx.se/mail/etiquette.html
bruce | 3 Jul 00:11 2016
Picon

curl issue...

Hi.

Trying to solve/figure out what I've screwed up in the following tw0 (2) curl statements.

Trying to get the results of the last curl, which should match the result of what I get when I insert the url in the browser with javascript disabled.

Pretty sure it's something subtle I've missed.

Thoughts/comments would be useful.

Thanks

echo '' > a.lwp
curl -vvv -A  "Mozilla/5.0 (iPad; CPU OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5355d Safari/8536.25"     --cookie-jar 'a.lwp' --cookie 'a.lwp'    -L "https://www.bkstr.com"

curl -vvv -A  "Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Firefox/38.0"     --cookie-jar "a.lwp" --cookie "a.lwp"  -e 'http://www.bkstr.com/montgomerycollegestore/shop/books/textbooks-and-course-materials'  -L "http://www.bkstr.com/webapp/wcs/stores/servlet/LocateCourseMaterialsServlet?requestType=INITIAL&storeId=432905&demoKey=d"


the results should be:
<script>parent.doneLoaded('{"meta":[{"request":"CAMPUSES","skip":"false","campusActive":"true","progActive":"true","termActive":"true","size":"3"}],"data":[{"Germantown Campus":"3202","Rockville Campus":"3200","Takoma Campus":"3201"}]}')</script>



-------------------------------------------------------------------
List admin: https://cool.haxx.se/list/listinfo/curl-users
FAQ:        https://curl.haxx.se/docs/faq.html
Etiquette:  https://curl.haxx.se/mail/etiquette.html
Attila-Mihaly Balazs | 1 Jul 15:15 2016
Picon

Curl might not be decompressing the body if the server sends it compressed without being asked to do so

I can't reliably reproduce this, but I think that if the server sends
the response compressed (ie. "Content-Encoding: gzip") without curl
asking for this (ie. the "--compressed" flag was not specified), curl
doesn't decompress the body.

I intermittently observed this behavior with some of Google's CDN
servers (ie. if you upload static files to google appengine it pushes
it to some "frontend caches" which seem to exhibit this behavior from
time to time).

My curl version is from the Ubuntu 16.04 package manager: curl 7.47.0
(x86_64-pc-linux-gnu) libcurl/7.47.0 GnuTLS/3.4.10 zlib/1.2.8
libidn/1.32 librtmp/2.3

This is nothing too major (and can be argued that the server
implementation is not conformant), but thought that somebody might be
interested.

Cheers,
Attila
-------------------------------------------------------------------
List admin: https://cool.haxx.se/list/listinfo/curl-users
FAQ:        https://curl.haxx.se/docs/faq.html
Etiquette:  https://curl.haxx.se/mail/etiquette.html
Gou Lingfeng | 28 Jun 14:05 2016
Picon

Segfault when downloading multiple sftp files in parallel

Dear Developers,

I used libcurl to download multiple files on a ssh server, in parallel with ``curl_multi_*``.
In the beginning 4 parallel jobs are run, and later only two parallel jobs are kept, and used for a long time.
Then time comes when 2 unused connections are pruned by ``prune_dead_connections``.
The new session data is associated to the old connections and ``sftp_disconnect`` is called just to modify the new session data (``Curl_safefree(conn->data->req.protop)``).
And later segfault happens when modifying conn->data->req.protop.

Here is the log (``Calloc/Free proto`` followed by ``conn``, ``conn->data`` and ``conn->data->req.protop``):

Calloc proto   0x7fffbc07fa10   0x7fffbc01a480   0x7fffbc07d980
* Found bundle for host 10.10.44.86: 0x7fffbc03f7f0 [serially]
* Re-using existing connection! (#4) with host 10.10.44.86
* Connection #5 to host 10.10.44.86 left intact
Calloc proto   0x7fffbc07fa10   0x7fffbc036270   0x7fffbc079660
* Found bundle for host 10.10.44.86: 0x7fffbc03f7f0 [serially]
* Re-using existing connection! (#5) with host 10.10.44.86
* Connection #5 to host 10.10.44.86 left intact
Calloc proto   0x7fffbc07faf0   0x7fffbc036270   0x7fffbc07bad0
* Connection 6 seems to be dead!
Free proto   0x7fffbc0422c0   0x7fffbc036270   0x7fffbc07bad0
* Closing connection 6
* Connection 7 seems to be dead!
Free proto   0x7fffbc040770   0x7fffbc036270 (nil)             
* Closing connection 7
* Found bundle for host 10.10.44.86: 0x7fffbc03f7f0 [serially]
* Re-using existing connection! (#5) with host 10.10.44.86

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffca9fe700 (LWP 3328)]
ssh_getworkingpath (homedir=0x7fffbc0795b0 "/home/goulf", path=path <at> entr
y=0x0,  
   conn=0x7fffbc0414f0, conn=0x7fffbc0414f0) at ssh.c:470
470       *path = real_path;


You can see that ``0x7fffbc07bad0`` is freed (twice).

Thanks,
glf
-------------------------------------------------------------------
List admin: https://cool.haxx.se/list/listinfo/curl-users
FAQ:        https://curl.haxx.se/docs/faq.html
Etiquette:  https://curl.haxx.se/mail/etiquette.html
CJ Ess | 28 Jun 03:21 2016
Picon

multiple POSTs via curl over http2

I'm trying to make multiple post requests with curl via http/2 - I'm using multiple sets of '-d "" url' or "-d <at> file url". For each request (and even with single requests) I see the message "HTTP 1.0, assume close after body" in the verbose output and each request is doing its own handshake and shutdown. The same command with --http1.1 instead of --http2 posts all five requests via the same connection. What do I need to do to get the same result in http/2?

-------------------------------------------------------------------
List admin: https://cool.haxx.se/list/listinfo/curl-users
FAQ:        https://curl.haxx.se/docs/faq.html
Etiquette:  https://curl.haxx.se/mail/etiquette.html
CJ Ess | 26 Jun 02:37 2016
Picon

Choosing http/1.1 over http/2 in Curl

I've built a new curl binary with http/2 support:

curl 7.49.1 (x86_64-pc-linux-gnu) libcurl/7.49.1 OpenSSL/1.0.2h nghttp2/1.11.1
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: IPv6 Largefile NTLM NTLM_WB SSL TLS-SRP HTTP2 UnixSockets

I've found that if the server offers h2 then curl uses it, regardless if --http2 or --no-http2 is present on the command line (neither option causes an error, but there is no difference in behavior).

There is a -0 option for http/1.0 (which I'm using as a workaround) but there is no option for http/1.1

Is there a method for forcing http/1.1 that maybe I've overlooked?

-------------------------------------------------------------------
List admin: https://cool.haxx.se/list/listinfo/curl-users
FAQ:        https://curl.haxx.se/docs/faq.html
Etiquette:  https://curl.haxx.se/mail/etiquette.html

Gmane