Re: CA bundle & life behind a firewall with proxy

Hi Quanah,
Am 31.03.2011 22:25, schrieb Quanah Gibson-Mount:
> I noticed that the step to make a CA bundle fails when you live behind a
> firewall and require the use of a proxy to get outside of the network,
> even if I have the various environment variables properly set.
>
> Fortunately, it is easy to fix this by patching the perl script thusly:
>
> quanah <at> zre-ldap001:~/p4/main/ThirdParty/curl/patches$ more proxy.patch
> --- curl-7.19.7/lib/mk-ca-bundle.pl.orig 2010-03-02 17:47:44.000000000
> -0800
> +++ curl-7.19.7/lib/mk-ca-bundle.pl 2010-03-02 17:48:01.000000000 -0800
>  <at>  <at>  -79,6 +79,7  <at>  <at> 
> if (!$opt_n || !-e $txt) {
> print "Downloading '$txt' ...\n" if (!$opt_q);
> my $ua = new LWP::UserAgent(agent => "$0/$version");
> + $ua->env_proxy();
> my $req = new HTTP::Request('GET', $url);
> my $res = $ua->request($req);
> if ($res->is_success) {
>
>
> It seems to me that this would be worthwhile to add into curl itself,
> since it shouldn't have any effect when there are no proxy variables set
> in the environment, but will do the right thing when they are.
your patch does not apply since the script was changed meanwhile exactly 
at this place .... 
I've added your modification, but I've only tested that it still works 
when no proxy env var is set - please verify that also works for you 
with proxy:

Re: CA bundle & life behind a firewall with proxy

--On Friday, April 01, 2011 3:11 PM +0200 Guenter <lists <at> gknw.net> wrote:

> Hi Quanah,
> Am 31.03.2011 22:25, schrieb Quanah Gibson-Mount:
>> I noticed that the step to make a CA bundle fails when you live behind a
>> firewall and require the use of a proxy to get outside of the network,
>> even if I have the various environment variables properly set.
>>
>> Fortunately, it is easy to fix this by patching the perl script thusly:
>>
>> quanah <at> zre-ldap001:~/p4/main/ThirdParty/curl/patches$ more proxy.patch
>> --- curl-7.19.7/lib/mk-ca-bundle.pl.orig 2010-03-02 17:47:44.000000000
>> -0800
>> +++ curl-7.19.7/lib/mk-ca-bundle.pl 2010-03-02 17:48:01.000000000 -0800
>>  <at>  <at>  -79,6 +79,7  <at>  <at> 
>> if (!$opt_n || !-e $txt) {
>> print "Downloading '$txt' ...\n" if (!$opt_q);
>> my $ua = new LWP::UserAgent(agent => "$0/$version");
>> + $ua->env_proxy();
>> my $req = new HTTP::Request('GET', $url);
>> my $res = $ua->request($req);
>> if ($res->is_success) {
>>
>>
>> It seems to me that this would be worthwhile to add into curl itself,
>> since it shouldn't have any effect when there are no proxy variables set
>> in the environment, but will do the right thing when they are.
> your patch does not apply since the script was changed meanwhile exactly
> at this place .... 
> I've added your modification, but I've only tested that it still works

Re: CA bundle & life behind a firewall with proxy

--On Friday, April 01, 2011 9:16 AM -0700 Quanah Gibson-Mount 
<quanah <at> zimbra.com> wrote:

>> I've added your modification, but I've only tested that it still works
>> when no proxy env var is set - please verify that also works for you with
>> proxy:
>> https://github.com/bagder/curl/blob/master/lib/mk-ca-bundle.pl
>
> Will do. It applies (with offset) fine for me anyhow to 7.21.4. ;)

I downloaded the copy from github, and it works for me.  Thanks!

[root <at> zre-rhel4 curl-7.21.4]# make ca-bundle
generate a fresh ca-bundle.crt
Downloading 'certdata.txt' ...
Processing 'certdata.txt' ...
[snip]
Done (165 CA certs processed).

--Quanah

--

Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra ::  the leader in open source messaging and collaboration
-------------------------------------------------------------------

HTTP POST: howto get just the the response content?

I'm new to libcurl so this might be obvious...

If I do a HTTP get with a
....
curl_easy_setopt(_curl, CURLOPT_WRITEFUNCTION, write_callback_std_stream);
curl_easy_setopt(_curl, CURLOPT_WRITEDATA, ostream);
....
then my callback "write_callback_std_stream" gets called with the
content of the request.

but when I try to do a HTTP POST my request content ends up correctly
at the server which the responds with content (a binary file)

however using the same construct as above

 ....
        curl_easy_setopt (_curl, CURLOPT_URL,url.c_str());
        curl_easy_setopt (_curl, CURLOPT_POST, 1);

        /* the transmitted content */
        curl_easy_setopt (_curl, CURLOPT_POSTFIELDSIZE_LARGE ,
(curl_off_t) content_lenght); // must be different in post and put???
        /* we want to use our own read function */
        curl_easy_setopt(_curl, CURLOPT_READFUNCTION, read_callback_std_stream);
        /* now specify which file to upload */
        curl_easy_setopt(_curl, CURLOPT_READDATA, istream);

        /* the reply */
        curl_easy_setopt(_curl, CURLOPT_WRITEFUNCTION,
write_callback_std_stream);

Re: HTTP POST: howto get just the the response content?

On Sat, 2 Apr 2011, svante karlsson wrote:

> I'm new to libcurl so this might be obvious...
>
> If I do a HTTP get with a
> ....
> curl_easy_setopt(_curl, CURLOPT_WRITEFUNCTION, write_callback_std_stream);
> curl_easy_setopt(_curl, CURLOPT_WRITEDATA, ostream);
> ....
> then my callback "write_callback_std_stream" gets called with the
> content of the request.

No, it gets called with the content of the *response*. The request is what you 
send to the server and that is not passed to the write callback.

> however using the same construct as above

> my callback gets called first with each response header then with the
> (my) desired content - while the headers are interesting is there a
> way of getting rid of those before the callback?

You should only get HTTP headers in the WRITEFUNCTION callback if you have 
CURLOPT_HEADER set.

--

-- 

  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html

Re: HTTP POST: howto get just the the response content?

>
> You should only get HTTP headers in the WRITEFUNCTION callback if you have
> CURLOPT_HEADER set.

Thanks, that solved the problem.

I had that options set (also for get and put)
  curl_easy_setopt(_curl, CURLOPT_VERBOSE, 1L);	
  curl_easy_setopt(_curl, CURLOPT_HEADER, 1L);

/svante
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html

Subject: Problem when cross compile libcurl(7.21.4) from source on mac

Problem when cross compile libcurl(7.21.4) from source on mac

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html

Re: Problem when cross compile libcurl(7.21.4) from source on mac

On Sat, 2 Apr 2011, 陈天舟 wrote:

> See `config.log' for more details

This is truly your best lead. The config.log file will contain a detailed 
description of what the script tried to do and what the result was. Based on 
that, you might get some clues on what to adjust to proceed.

If not, consider showing us the part of the config.log that includes that 
error and we might be able to offer some ideas.

--

-- 

  / daniel.haxx.se

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html

how to filter redirections

Hi,
I have to track some sites using curlpp. When i download, some site open
popup coming from ads or porno servers. 
I'd like to block those popup telling curl to follow redirections, but
either to blacklist some sites, or to allow only redirection in the
asked site.
Is it possible ?
thanks for your help.

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html

Re: potential TFTP bug

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html