Vinders, Paul | 1 Sep 2006 08:52
Picon

AW: AW: cURL on VMS: lost linefeeds in asciifiles

The problem seems to be solved if a file on OpenVMS is created with RecordType = Stream_LF instead of Fixed or
Variable (in Fortran). The File contains '0A'X after every line instead of the VMS-specific lines which
start with the bytes for 'length of the line' implicitly meaning CR/LF. Also files created with the
EDT-editor have these structure, so they cannot be transferred in a proper way too. I'll try to solve this problem.

Sincerely,
Paul

-----Ursprüngliche Nachricht-----
Von: curl-library-bounces <at> cool.haxx.se [mailto:curl-library-bounces <at> cool.haxx.se] Im Auftrag
von Daniel Stenberg
Gesendet: Donnerstag, 31. August 2006 13:11
An: libcurl development
Betreff: Re: AW: cURL on VMS: lost linefeeds in asciifiles

On Thu, 31 Aug 2006, Vinders, Paul wrote:

> No, i'm using the library, not the command line tool. But in my 
> C-program I've set the Ftp-Ascii option already.

FTP ASCII conversion support was added very recently and as I said, it does look like it isn't yet complete.
Feel free to debug it and provide patches.

--
  Commercial curl and libcurl Technical Support: http://haxx.se/curl.html

Daniel Stenberg | 1 Sep 2006 09:15
Picon
Favicon
Gravatar

Re: large number of files transfer failed

On Thu, 31 Aug 2006, Parvinder Singh Arora wrote:

> I had initiated a million file transfer using libcurl but it got aborted in
> between, gave following errors

Congratulations, this is one of the most vague reports we ever got.

Care to elaborate a lot exactly what you have and what you do and what you 
see?

--

-- 
  Commercial curl and libcurl Technical Support: http://haxx.se/curl.html

Tom Jerry | 1 Sep 2006 19:04
Picon

does curl_multi handle can be accessed from 2 threads?

Hello,
I'm new to this library and I consider using it in my application.
What my application (service) should to is having *several* clients that are always ready to recieve http requests, execute them and return the answer.
I was looking at some multi examples, and currently what I'm thinking to do is a s follows:

I will have 2 threads:

1st thread is initializing the multi handle and running in endless loop doing the following (more or less):

  while (true)
  {
    curl_multi_fdset(multi_handle, &fdread, &fdwrite, &fdexcep, &maxfd);

    rc = select(maxfd+1, &fdread, &fdwrite, &fdexcep, &timeout);
 
    while(CURLM_CALL_MULTI_PERFORM ==
            curl_multi_perform(multi_handle, &still_running));
  }

I will have of course my own write function, and notify the 2nd thread (see below) upon transfer completion.


2nd thread is my main thread. It will generally wait on a queue for messages.
Those messages may be a new coming request or a notification of a completion of a finished request (see above).
Upon a new request, I need to execute it. So I will initialize an easy handle (or reuse an old one).
The question is:
can I use
   curl_multi_add_handle(multi_handle, http_handle);
On this thread for adding new requests to process?
Is it thread safe?

If not, can you instruct me how to keep adding new requests in one thread and still use a select on another thread?

Thank you,

Ori.


Daniel Stenberg | 1 Sep 2006 20:17
Picon
Favicon
Gravatar

Re: does curl_multi handle can be accessed from 2 threads?

On Fri, 1 Sep 2006, Tom Jerry wrote:

> The question is:
> can I use
>  curl_multi_add_handle(multi_handle, http_handle);
> On this thread for adding new requests to process?
> Is it thread safe?

It is thread-safe, but you can only use the single multi handle in one thread 
at a time, not simultanouesly.

--

-- 
  Commercial curl and libcurl Technical Support: http://haxx.se/curl.html

Tom Jerry | 1 Sep 2006 22:47
Picon

Re: does curl_multi handle can be accessed from 2 threads?

So will it help if I lock the multi handle before accessing it on both threads?
Is there any other way of writing the service I want?


On 9/1/06, Daniel Stenberg <daniel <at> haxx.se> wrote:
On Fri, 1 Sep 2006, Tom Jerry wrote:

> The question is:
> can I use
>  curl_multi_add_handle(multi_handle, http_handle);
> On this thread for adding new requests to process?
> Is it thread safe?

It is thread-safe, but you can only use the single multi handle in one thread
at a time, not simultanouesly.
 


Daniel Stenberg | 1 Sep 2006 22:52
Picon
Favicon
Gravatar

Re: does curl_multi handle can be accessed from 2 threads?

On Fri, 1 Sep 2006, Tom Jerry wrote:

> So will it help if I lock the multi handle before accessing it on both 
> threads?

lock? It will work if you have a mutex around your use of the multi handle, 
sure.

--

-- 
  Commercial curl and libcurl Technical Support: http://haxx.se/curl.html

Tom Jerry | 2 Sep 2006 10:58
Picon

Re: does curl_multi handle can be accessed from 2 threads?

Yes, that's what I ment, a mutex.
I will try that. thanks.


On 9/1/06, Daniel Stenberg < daniel <at> haxx.se> wrote:
On Fri, 1 Sep 2006, Tom Jerry wrote:

> So will it help if I lock the multi handle before accessing it on both
> threads?

lock? It will work if you have a mutex around your use of the multi handle,
sure.

--
  Commercial curl and libcurl Technical Support: http://haxx.se/curl.html

Daniel Stenberg | 2 Sep 2006 13:04
Picon
Favicon
Gravatar

Re: patch to add CURLINFO_ERRROBUFFER

On Tue, 29 Aug 2006, Mohun Biswas wrote:

>> Can't you store the errorbuffer pointer in a private struct and link it to
>> the handle with CURLOPT_PRIVATE instead?
>
> Well, yes, but let me recast the issue. On further thought, this is the sort 
> of thing that belongs in a curl_easy_getopt() function, not 
> curl_easy_getinfo.

Quite true. CURLOPT_PRIVATE was mostly added to curl_easy_getinfo() to save us 
from having to add a new function.

> So I did a little web searching and found that it's come up before (cf 
> <http://curl.haxx.se/mail/lib-2005-09/0246.html>). In that thread, Daniel 
> was reluctant to allow curl_easy_getopt in without a compelling argument for 
> it. It seems to me that the requirement to free our own data is the 
> compelling argument, because there are a number of values (URLs, error 
> buffers, slists, etc) which have to be set into the easy handle with 
> curl_easy_setopt and which we (API users) have the responsibility of freeing 
> later, and quite possibly in a different stack frame.

I'm not convinced that is a compelling argument. You still need to know 
exactly which of these values to extract and how to free each one of them, 
which thus forces you to pass logic around for that and if you do that, having 
the actual pointers stored too is hardly much extra work.

> So yes, we could invent our own structures containing lists of pointers to 
> free and pass those around using CURLOPT_PRIVATE. But to my mind, 
> CURLOPT_PRIVATE is an escape hatch analogous to eval in Perl or being able 
> to insert assembler code into C code - it's good to have the option for 
> special situations but if something is being done predictably and 
> repeatedly, it's better to add true support for it.

I'll explain how I see it: the curl objects are opaque objects to 
applications. Apps have no idea what they contain or not, and CURLOPT_PRIVATE 
is just a way to attach a private sticker on this big barrel of unknown 
contents. You can read that sticker again later but you still have no real 
clue what the barrel contains.

I have no understanding for the parallells with eval in Perl or asm keywords 
in C. I just don't see how they are anywhere close to this.

> And finally, I find the argument made in the referenced thread persuasive 
> too. It is that a curl easy handle is basically an object with a bunch of 
> attributes. The great majority of the SW world has come to agree that the 
> setter/getter paradigm is valuable, so what's the argument for libcurl 
> offering setters but no equivalent getters? We can always keep track of the 
> data we set into the object but that basically means two copies with all the 
> opportunities for skew and added complexity that entails.

So then you're now saying we should allow some 135 "attributes" to be readable 
from the easy handle? What good would that bring the majority of libcurl 
users?

I'm still reluctant to add things to libcurl that

  1 - easily can be provided and done by the application itself and
  2 - that won't used by most libcurl applications and
  3 - that will grow the library if done "right"

Of course the discussion is what features that would fit this description and 
what features that don't.

In general, when I'm leaning towards a 'no' for including something I want to 
see what others have to say about the suggestion and if then there's very 
little support or talks about it, I consider that in favour of my way...

So, is there anyone else around with opinions in this matter? Be it in favour 
or against, I'm interested.

--

-- 
  Commercial curl and libcurl Technical Support: http://haxx.se/curl.html

Mohun Biswas | 2 Sep 2006 16:02
Favicon

Re: patch to add CURLINFO_ERRROBUFFER

Daniel Stenberg wrote:
> I'm not convinced that is a compelling argument. You still need to know 
> exactly which of these values to extract and how to free each one of 
> them, which thus forces you to pass logic around for that and if you do 
> that, having the actual pointers stored too is hardly much extra work.

This part doesn't make sense to me. Quite possibly other people program 
libcurl differently but at least in my case, and I'd imagine in many 
others, these things are a matter of policy. In the original use case, I 
allocate an error buffer for every easy handle so there's no trick to 
knowing that it must be freed subsequently. There's no need to "pass 
logic around" - merely to have access to the pointer at cleanup time.

I'm not necessarily arguing for a curl_easy_getopt - my own view is more 
limited. It just seems inelegant that I have to invent my own struct - 
or list of structs or struct of lists or whatever, because I use 
CURLOPT_PRIVATE for other things too - simply to pass along *copies* of 
pointers which the libcurl API already knows about and expects me to 
free. It just seems cleaner to ask it "what was that error buffer's 
address again?"

An alternate solution might be found by comparison with curl_formadd 
which gives the choice of PTR (you free it) or COPY (we free it for you) 
pointers. But that's another dimension of complexity.

Or one could imagine a way of registering pointers to be freed at 
cleanup time:

	char *errbuf = calloc(CURL_ERROR_SIZE, 1);
	curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, errbuf);
	curl_easy_setopt(curl, CURLOPT_FREE_AT_CLEANUP, errbuf);

which might have the smallest code and doc impact.

> Of course the discussion is what features that would fit this 
> description and what features that don't.
> 
> In general, when I'm leaning towards a 'no' for including something I 
> want to see what others have to say about the suggestion and if then 
> there's very little support or talks about it, I consider that in favour 
> of my way...
> 
> So, is there anyone else around with opinions in this matter? Be it in 
> favour or against, I'm interested.

Likewise.

MB

Diniz Bortolotto | 3 Sep 2006 04:14
Picon

HTTP suport disabled

Hi!

I am new here. I subscribed the curl-library Mailing List because I
have a problem.

I tried to use curl and received this message:
* libcurl was built with HTTP disabled, http: not supported!
curl: (1) libcurl was built with HTTP disabled, http: not supported!

and...

The "curl -V" command response:
curl 7.15.5 (i686-pc-linux-gnu) libcurl/7.12.2 zlib/1.2.3
Protocols: ftp
Features: IPv6 Largefile libz

Somebody can help me?

Reggards!

--

-- 
Diniz Fernando Bortolotto Ferreira
Analista de Informática
Brasília - DF


Gmane