headers
Konstantin Schauwecker | 1 Jun 2003 11:41
Picon

multithreading problem with libcurl

Hello everybody

I have some strange problems when using libcurl in multiple threads. I've 
spend days of debugging and modifying my source, and now I'm completely 
helpless :-(
I have an application which uses 10 threads (I had the problem with lower 
counts of threads too). Each thread calls a curl_easy_init to get a curl 
handle. All threads are doing constantly HTTP requests. 

Now my Problems:

On Linux:
The program runs fine for a few minutes, but then it crashes with a 
segmentation fault. Valgrind told me, that the reason for crashing is inside 
of libcurl. Here the output of valgrind:

==1487== valgrind's libpthread.so: KLUDGED call to: siglongjmp (cleanup 
handlers are ignored)
==1487== Invalid read of size 4
==1487==    at 0x40235303: Curl_resolv (hostip.c:242)
==1487==    Address 0x41B0C3B0 is on thread 1's stack
==1487==
==1487== Invalid read of size 4
==1487==    at 0x4023B01B: Curl_failf (sendf.c:154)
==1487==    Address 0x10 is not stack'd, malloc'd or free'd

I thougth this might be a problem of setting the wrong options. So I tried to 
remove the calls of curl_easy_setopt to a minimum  (I already tried reducing 
all code to a minimum *g*), but it didn't help.
(Continue reading)

Permalink | Reply | 5 comments |
headers
Jean-Philippe Barrette-LaPierre | 1 Jun 2003 19:36

curl's doc web page error

just above the description of the option CURLINFO_NAMELOOKUP_TIME for the 
function curl_easy_getinfo, there's missing those lines:

       CURLINFO_CONNECT_TIME
               Pass  a pointer to a double to receive the time, in seconds, it
               took from the start until the connect to the  remote  host  (or
               proxy) was completed.

       CURLINFO_PRETRANSFER_TIME
               Pass  a pointer to a double to receive the time, in

-------------------------------------------------------
This SF.net email is sponsored by: eBay
Get office equipment for less on eBay!
http://adfarm.mediaplex.com/ad/ck/711-11697-6916-5
Permalink | Reply | 1 comment |
headers
Jean-Philippe Barrette-LaPierre | 1 Jun 2003 19:40

Re: multithreading problem with libcurl

On Sunday 01 June 2003 05:41 am, Konstantin Schauwecker wrote:
> Hello everybody
>
> I have some strange problems when using libcurl in multiple threads. I've
> spend days of debugging and modifying my source, and now I'm completely
> helpless :-(
> I have an application which uses 10 threads (I had the problem with lower
> counts of threads too). Each thread calls a curl_easy_init to get a curl
> handle. All threads are doing constantly HTTP requests.
>
> Now my Problems:
>
> On Linux:
> The program runs fine for a few minutes, but then it crashes with a
> segmentation fault. Valgrind told me, that the reason for crashing is
> inside of libcurl. Here the output of valgrind:

which version of curl are you using?

>
> ==1487== valgrind's libpthread.so: KLUDGED call to: siglongjmp (cleanup
> handlers are ignored)
> ==1487== Invalid read of size 4
> ==1487==    at 0x40235303: Curl_resolv (hostip.c:242)
> ==1487==    Address 0x41B0C3B0 is on thread 1's stack
> ==1487==
> ==1487== Invalid read of size 4
> ==1487==    at 0x4023B01B: Curl_failf (sendf.c:154)
> ==1487==    Address 0x10 is not stack'd, malloc'd or free'd
>
(Continue reading)

Permalink | Reply | 1 comment |
headers
Konstantin Schauwecker | 1 Jun 2003 20:20
Picon

Re: multithreading problem with libcurl

Am Sonntag, 1. Juni 2003 19:40 schrieb Jean-Philippe Barrette-LaPierre:
> which version of curl are you using?

I am using Version 7.10.5 on both systems.

-------------------------------------------------------
This SF.net email is sponsored by: eBay
Get office equipment for less on eBay!
http://adfarm.mediaplex.com/ad/ck/711-11697-6916-5
Permalink | Reply | 0 comments |
headers
Bo Lorentsen | 2 Jun 2003 08:38
Picon

Re: multithreading problem with libcurl

On Sun, 2003-06-01 at 11:41, Konstantin Schauwecker wrote:

> I have an application which uses 10 threads (I had the problem with lower 
> counts of threads too). Each thread calls a curl_easy_init to get a curl 
> handle. All threads are doing constantly HTTP requests. 
I have a simular setup (only 5 threads), but not the problems you
descripe here, not that it will help you :-)

> Now my Problems:
> 
> On Linux:
> The program runs fine for a few minutes, but then it crashes with a 
> segmentation fault. Valgrind told me, that the reason for crashing is inside 
> of libcurl. Here the output of valgrind:
One silly question, do you handle timeouts ?, and have you then disabled
the signal handler (CURLOPT_NOSIGNAL) ? 

Would it be possible for you to post both some sample code, and a small
Makefile that generated the crash ?

/BL

-------------------------------------------------------
This SF.net email is sponsored by: eBay
Get office equipment for less on eBay!
http://adfarm.mediaplex.com/ad/ck/711-11697-6916-5
Permalink | Reply | 2 comments |
headers
Daniel Stenberg | 2 Jun 2003 09:21
Picon
Favicon
Gravatar

Re: curl_debug_callback

On Wed, 28 May 2003, Robert Smith wrote:

> Does anyone have an example of the debug callback function?  I couldn't
> find one on the website.

The curl command line tool uses the debug function and you can see how in the
src/main.c function:

Set at:

	http://curl.haxx.se/lxr/source/src/main.c#L2967

Function present at:

	http://curl.haxx.se/lxr/source/src/main.c#L2224

I hope this helps. Please just ask again if not.

--

-- 
 Daniel Stenberg -- curl: been grokking URLs since 1998

-------------------------------------------------------
This SF.net email is sponsored by: eBay
Get office equipment for less on eBay!
http://adfarm.mediaplex.com/ad/ck/711-11697-6916-5
Permalink | Reply | 0 comments |
headers
Daniel Stenberg | 2 Jun 2003 09:54
Picon
Favicon
Gravatar

Re: Properly free()'ing memory after using CURLOPT_WRITEFUNCTION

On Fri, 30 May 2003, Joel  Apter wrote:

> After looking at the example given in docs/examples/getinmemory.c in the
> curl directory, I was wondering what the proper syntax and usage of free()
> would be to release this memory efficiently after it has been used
> (free(chunk.memory); ?).
>
> I am not very familiar with this type of memory handling in C.  Thank you
> for your help.

Correct, to free the memory allocated by the write callback used in that
example source code, you should add a line that does something like this:

	free(chunk.memory);

--

-- 
 Daniel Stenberg -- curl: been grokking URLs since 1998

-------------------------------------------------------
This SF.net email is sponsored by: eBay
Get office equipment for less on eBay!
http://adfarm.mediaplex.com/ad/ck/711-11697-6916-5
Permalink | Reply | 0 comments |
headers
Daniel Stenberg | 2 Jun 2003 09:45
Picon
Favicon
Gravatar

RE: memory leak or my error?

On Thu, 29 May 2003, Paul Accosta wrote:

> Ive pretty much combed the entire program and used Purify to make sure
> there is no memory leaks on my part. I do a check by getting the memroy
> before these series of calls and then get a check of memory after clean_up
> and it is still higher.

But... if you have Purify and can use that, why can't you just use it and see
if libcurl leaks anything or not? (I haven't ever used that myself.)

I'm pretty confident that libcurl doesn't leak anything in most "normal"
cases, as I always run the test suite with full memory leak detection
enabled, but there might of course exist corner-cases that we haven't added
to the test suite and thus remain undetected.

> The program never terminitnates so I do not use curl_global_cleanup.

You shouldn't get any leaks because of that.

> Im not sure what else I can do, any ideas?

Have you read the lib/README.memoryleak document?

Oh, and make sure that you use a recent version of libcurl.

--

-- 
 Daniel Stenberg -- curl: been grokking URLs since 1998

-------------------------------------------------------
This SF.net email is sponsored by: eBay
(Continue reading)

Permalink | Reply | 0 comments |
headers
Daniel Stenberg | 2 Jun 2003 11:24
Picon
Favicon
Gravatar

Re: SSL session resume problem?

On Fri, 30 May 2003, Cyrill Osterwalder wrote:

> The only reason why I keep my curl handle at this point is because I'd like
> to benefit from SSL session resumes. I'm testing the SSL handshakes and
> find that the SSL sessions are not resumed if I create new curl handles so
> reusing the handle looks like a must.

Correct. That is the only way libcurl supports session ID caching/re-use.

> However, reusing the curl handle does not seem to provide 100% SSL session
> resume support. According to the OpenSSL trace log of the web server,
> libcurl does not seem to update it's SSL session

Can you provide us with a public URL and example code showing this happen?

> - if it is not a new handshake but an attempt to resume the SSL session
>
> - if the server does not resume the SSL session for any reason (SSL session
> dead, cache miss, etc)

libcurl re-uses the session ID if it has one in its cache for the same name,
and it has no existing TCP connection to the site (if it has it re-uses that
instead).

That's the theory at least.

> In this case, a new SSL session is created between client and server but it
> does not seem to be kept by the libcurl client. This happens now for each
> following requests. I can provide the server SSL engine log files if
> anybody would be interested.
(Continue reading)

Permalink | Reply | 1 comment |
headers
Daniel Stenberg | 2 Jun 2003 11:28
Picon
Favicon
Gravatar

Re: Advise and Information request

On Fri, 30 May 2003, Sinha, Raj (Raj) wrote:

(skipping the info Dan already provided)

> Can anyone tell me if they have successfully ported to VxWorks. ( Imagine
> it should be easy since they has been successful ports to Unix)

I haven't heard anyone doing this, but if you use the POSIX layer (I recall
VxWorks having one of those) I can't see how this should be hard.

If you need to make any adjustments, I'll be interested in getting a patch
back!

--

-- 
 Daniel Stenberg -- curl: been grokking URLs since 1998

-------------------------------------------------------
This SF.net email is sponsored by: eBay
Get office equipment for less on eBay!
http://adfarm.mediaplex.com/ad/ck/711-11697-6916-5
Permalink | Reply | 0 comments |

Gmane