Mailing list for users of libevent ()

Kenneth Cox | 2 Feb 2010 18:06

evhttp_make_request() fails, but leaves request queued

I am using libevent-1.4.12-stable on Linux, and occasionally 
getaddrinfo() fails when called from evhttp_connection_connect() called 
from evhttp_make_request().  When this happens, the request remains 
queued on the connection.  So sometime later when another request is 
made, getaddrinfo() succeeds, and the request gets dispatched, late.

My question is, how should this ideally be handled?  It sounds to me 
like a shortcoming of the libevent http api, but before I suggest 
patches I would like to hear other opinions.

The http request is not useful to me if it's delayed.  And yet I rely on 
the request callback to free some memory associated with the request.

Regards,
Ken

headers

Niels Provos | 2 Feb 2010 18:27

Re: evhttp_make_request() fails, but leaves request queued

On Tue, Feb 2, 2010 at 9:06 AM, Kenneth Cox <kenstir <at> gmail.com> wrote:
> The http request is not useful to me if it's delayed.  And yet I rely on
> the request callback to free some memory associated with the request.

You could resolve the address yourself and then issue the request.
At that point, if the connection fails and no retries have been
specified, it errors out immediately.

Niels.

headers

David Titarenco | 3 Feb 2010 03:18

evhttp_send_reply segfaults and aborts randomly

I wrote a high performance HTTP event server in C++ using libevent 2.0.1-alpha. The server isn't that complex so I'll try to explain what it does and whether or not this is a bug in libevent or a problem with my code.

A "bead" is basically a client connection. It looks like so:
struct bead {
    long int id;
    int comet;
    struct event *timeout_timer;

    int    lastMessage;
    int    delta;
    string whoAmI;
    string currCircle;
    string userHash;
    string cookieHash;
    string stickyHash;
    string appHash;
    bead() : whoAmI(""), currCircle(""), userHash(""), cookieHash(""), appHash("")   {}

};

It's mapped against the appropriate request pointer at it's initialization:
map<evhttp_request *,bead> rMap;

In main():
evhttp_set_gencb(http_server, generic_request_handler, NULL);

In generic_request_handler():
void generic_request_handler(struct evhttp_request *req, void *arg) {
    const char *ruri = evhttp_request_get_uri(req);

    if (strncmp(ruri, "/admin.eve", 10) == 0) {
        struct evbuffer *evb = evbuffer_new();
        evbuffer_add_printf(evb, "<h2>Hello. I am Eve.</h2><br> \
                Connections since start: %i <br> \
                Current active connections in STL container: %i <br>", (int) num_con, (int)rMap.size());
        evhttp_send_reply(req, HTTP_OK, "OK", evb);
        evbuffer_free(evb);
    }
    else
        long_polling_handler(req, NULL);
}

and lets look at the long_polling_handler():
void long_polling_handler(struct evhttp_request *req, void *arg) {
    // tons of code that handles POST variables pertinent to my app, creates new_bead, sets a timeout timer dependent on delta, etc, etc.
    rMap.insert(std::pair<evhttp_request *,bead>(req, new_bead));
}

The program loops over all the beads like so:
void handle_beads() {
    for( map<evhttp_request *,bead>::iterator it=rMap.begin(); it != rMap.end(); ++it)
    {
        long_polling_bead(it);
    }
}

static void iterate_cb(int fd, short event, void *arg) {
    handle_beads();
    event_free(evtq_timer);
    evtq_timer = evtimer_new(base, iterate_cb, NULL);
    evutil_timerclear(&tv);
    tv.tv_usec = 100000;
    event_add(evtq_timer, &tv);
    return;
}

The iteration callback is called every 100000 microseconds.

And finally, long_polling_bead(). This is where the program breaks...
void long_polling_bead(map<evhttp_request *,bead>::iterator it) {
// lots of code that connects to an SQL database, compares the user state (from POST) to the server state (from SQL)
// here, we either return 503 or 504
// 503 means NOT_UPDATED
// 504 tells nginx it should hit PHP (my server acts as an HTTP event proxy)
// but this is how I finish the requests:

            struct evbuffer *evb = evbuffer_new();
            evbuffer_add_printf(evb, "{\"error\":\"Some JSON stuff here\"}");
            evhttp_send_reply((*it).first,503,"ERROR",evb);
            evbuffer_free(evb);
            event_free((*it).second.timeout_timer);
            pLog->Write("Some log message");
            rMap.erase(it);
            return;

}

Here are the gdb backtraces:

#0 0xb7c739b4 in free () from /lib/libc.so.6
#1 0xb7f44b18 in mm_free (ptr=0x1) at event.c:2065
#2 0xb7f579b6 in evhttp_response_code (req=0xfffffff9, code=503, reason=0x813d2c8 "ERROR") at http.c:2124
#3 0xb7f5cd23 in evhttp_send_reply (req=0x98f9ba0, code=503, reason=0x813d2c8 "ERROR", databuf=0x991efb0) at http.c:2053
#4 0x0811911b in long_polling_bead (it={_M_node = 0xb7d4d170}) at eve.new.cpp:386
#5 0x08119cb1 in handle_beads () at eve.new.cpp:477
#6 0x08119d02 in iterate_cb (fd=-1, event=1, arg=0x0) at eve.new.cpp:482
#7 0xb7f47f18 in event_base_loop (base=0x98ce9c0, flags=0) at event.c:925
#8 0xb7f48c85 in event_base_dispatch (event_base=0x98ce9c0) at event.c:1005
#9 0x08118cbf in main (argc=1, argv=0xbf84b514) at eve.new.cpp:622

#0 0xb7fdb7f2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2
#1 0xb7c93df0 in raise () from /lib/libc.so.6
#2 0xb7c95701 in abort () from /lib/libc.so.6
#3 0xb7fb5447 in event_exit (errcode=0) at log.c:77
#4 0xb7fb5476 in event_errx (eval=-559030611, fmt=0xb7fcb170 "%s:%d: Assertion %s failed in %s") at log.c:134
#5 0xb7fbddbc in evhttp_send_reply (req=0x9e4f5a0, code=503, reason=0x813d2c8 "ERROR", databuf=0x9e04648) at http.c:2037
#6 0x0811911b in long_polling_bead (it={_M_node = 0xb7dae170}) at eve.new.cpp:386
#7 0x08119cb1 in handle_beads () at eve.new.cpp:477
#8 0x08119d02 in iterate_cb (fd=-1, event=1, arg=0x0) at eve.new.cpp:482
#9 0xb7fa8f18 in event_base_loop (base=0x9dec9c0, flags=0) at event.c:925
#10 0xb7fa9c85 in event_base_dispatch (event_base=0x9dec9c0) at event.c:1005
#11 0x08118cbf in main (argc=1, argv=0xbfd8d4b4) at eve.new.cpp:622

I hope I was thorough enough, thanks for any help.

- David Titarenco

headers

Niels Provos | 4 Feb 2010 00:12

Re: after the http client abruptly closes socket libevent (2.0.3-alpha) http server crashes

On Sun, Jan 31, 2010 at 6:57 AM, Rui Lopes <rgl <at> ruilopes.com> wrote:
> But if it abruptly closes the socket, the libevent code crashes
> somewhere inside the evhttp_send_reply_chunk function:

This is fixed in the 2.0 branch of the git repository now.   It should
probably be back-ported, too.

Niels.
***********************************************************************
To unsubscribe, send an e-mail to majordomo <at> freehaven.net with
unsubscribe libevent-users    in the body.

headers

Brodie Thiesfield | 4 Feb 2010 03:58

patch for libevent 1.4.13 for building win32 + unicode

Hi,

The following (and attached) patch fixes calls to the win32 api to 
explicitly call the char* versions of the functions. This fixes build 
failures when libevent is built with the UNICODE define.

Regards,
Brodie

--- evdns.orig.c	2009-07-24 00:17:02.000000000 +0900
+++ evdns.c	2010-02-04 11:46:54.046659000 +0900
 <at>  <at>  -2848,3 +2848,3  <at>  <at> 

-	if (!(handle = LoadLibrary("iphlpapi.dll"))) {
+	if (!(handle = LoadLibraryA("iphlpapi.dll"))) {
  		log(EVDNS_LOG_WARN, "Could not open iphlpapi.dll");
 <at>  <at>  -2918,3 +2918,3  <at>  <at> 

-	if (RegQueryValueEx(key, subkey, 0, &type, NULL, &bufsz)
+	if (RegQueryValueExA(key, subkey, 0, &type, NULL, &bufsz)
  	    != ERROR_MORE_DATA)
 <at>  <at>  -2924,3 +2924,3  <at>  <at> 

-	if (RegQueryValueEx(key, subkey, 0, &type, (LPBYTE)buf, &bufsz)
+	if (RegQueryValueExA(key, subkey, 0, &type, (LPBYTE)buf, &bufsz)
  	    == ERROR_SUCCESS && bufsz > 1) {
 <at>  <at>  -2954,3 +2954,3  <at>  <at> 

-		if (RegOpenKeyEx(HKEY_LOCAL_MACHINE, WIN_NS_NT_KEY, 0,
+		if (RegOpenKeyExA(HKEY_LOCAL_MACHINE, WIN_NS_NT_KEY, 0,
  				 KEY_READ, &nt_key) != ERROR_SUCCESS) {
 <at>  <at>  -2959,3 +2959,3  <at>  <at> 
  		}
-		r = RegOpenKeyEx(nt_key, "Interfaces", 0,
+		r = RegOpenKeyExA(nt_key, "Interfaces", 0,
  			     KEY_QUERY_VALUE|KEY_ENUMERATE_SUB_KEYS,
 <at>  <at>  -2974,3 +2974,3  <at>  <at> 
  		HKEY win_key = 0;
-		if (RegOpenKeyEx(HKEY_LOCAL_MACHINE, WIN_NS_9X_KEY, 0,
+		if (RegOpenKeyExA(HKEY_LOCAL_MACHINE, WIN_NS_9X_KEY, 0,
  				 KEY_READ, &win_key) != ERROR_SUCCESS) {
--- sample\event-test.orig.c	2008-11-17 05:35:40.000000000 +0900
+++ sample\event-test.c	2010-02-04 11:49:56.751109100 +0900
 <at>  <at>  -75,3 +75,3  <at>  <at> 
  	// Open a file.
-	socket = CreateFile("test.txt",     // open File
+	socket = CreateFileA("test.txt",     // open File
  			GENERIC_READ,                 // open for reading

--- evdns.orig.c	2009-07-24 00:17:02.000000000 +0900
+++ evdns.c	2010-02-04 11:46:54.046659000 +0900
 <at>  <at>  -2848,3 +2848,3  <at>  <at> 

-	if (!(handle = LoadLibrary("iphlpapi.dll"))) {
+	if (!(handle = LoadLibraryA("iphlpapi.dll"))) {
 		log(EVDNS_LOG_WARN, "Could not open iphlpapi.dll");
 <at>  <at>  -2918,3 +2918,3  <at>  <at> 

-	if (RegQueryValueEx(key, subkey, 0, &type, NULL, &bufsz)
+	if (RegQueryValueExA(key, subkey, 0, &type, NULL, &bufsz)
 	    != ERROR_MORE_DATA)
 <at>  <at>  -2924,3 +2924,3  <at>  <at> 

-	if (RegQueryValueEx(key, subkey, 0, &type, (LPBYTE)buf, &bufsz)
+	if (RegQueryValueExA(key, subkey, 0, &type, (LPBYTE)buf, &bufsz)
 	    == ERROR_SUCCESS && bufsz > 1) {
 <at>  <at>  -2954,3 +2954,3  <at>  <at> 

-		if (RegOpenKeyEx(HKEY_LOCAL_MACHINE, WIN_NS_NT_KEY, 0,
+		if (RegOpenKeyExA(HKEY_LOCAL_MACHINE, WIN_NS_NT_KEY, 0,
 				 KEY_READ, &nt_key) != ERROR_SUCCESS) {
 <at>  <at>  -2959,3 +2959,3  <at>  <at> 
 		}
-		r = RegOpenKeyEx(nt_key, "Interfaces", 0,
+		r = RegOpenKeyExA(nt_key, "Interfaces", 0,
 			     KEY_QUERY_VALUE|KEY_ENUMERATE_SUB_KEYS,
 <at>  <at>  -2974,3 +2974,3  <at>  <at> 
 		HKEY win_key = 0;
-		if (RegOpenKeyEx(HKEY_LOCAL_MACHINE, WIN_NS_9X_KEY, 0,
+		if (RegOpenKeyExA(HKEY_LOCAL_MACHINE, WIN_NS_9X_KEY, 0,
 				 KEY_READ, &win_key) != ERROR_SUCCESS) {
--- sample\event-test.orig.c	2008-11-17 05:35:40.000000000 +0900
+++ sample\event-test.c	2010-02-04 11:49:56.751109100 +0900
 <at>  <at>  -75,3 +75,3  <at>  <at> 
 	// Open a file. 
-	socket = CreateFile("test.txt",     // open File 
+	socket = CreateFileA("test.txt",     // open File 
 			GENERIC_READ,                 // open for reading

headers

Nick Mathewson | 4 Feb 2010 06:00

Re: patch for libevent 1.4.13 for building win32 + unicode

On Wed, Feb 3, 2010 at 9:58 PM, Brodie Thiesfield
<brofield2 <at> jellycan.com> wrote:
> Hi,
>
> The following (and attached) patch fixes calls to the win32 api to
> explicitly call the char* versions of the functions. This fixes build
> failures when libevent is built with the UNICODE define.

I've applied this to the 1.4 branch and to the 2.0 branch.  I also
tried building 2.0 with -DUNICODE and fixed the issues that turned up
in a more-or-less indiscriminate way.

Thanks,
--

-- 
Nick
***********************************************************************
To unsubscribe, send an e-mail to majordomo <at> freehaven.net with
unsubscribe libevent-users    in the body.

headers

Denis Bilenko | 5 Feb 2010 08:48

gevent 0.12.0 released

gevent is a coroutine-based Python networking library that uses
greenlet to provide a high-level synchronous API on top of libevent
event loop.

Features include:

- convenient API around greenlets
- familiar synchronization primitives (Event, Queue)
- cooperative socket and ssl modules
- WSGI server on top of libevent-http
- DNS requests done through libevent-dns
- monkey patching utility to get pure Python modules, like urllib2, to cooperate

Homepage: http://www.gevent.org
Download page: http://pypi.python.org/pypi/gevent

* changes in 0.12.0 *

- The major new feature is a gevent.ssl module, that provides
cooperative implementation of the standard ssl module. It does not
require any additional extensions on Python ≥ 2.6. It also works on
2.4 and 2.5 if ssl package is installed. The old, PyOpenSSL-based
implementation of SSL objects is still available, but the new version
is the preferred way now.
- The library now compiles and passes most of the relevant tests on
Windows. It’s still has a few rough edges (e.g. Ctrl-C is not
working), so it should be considered experimental.
- The socket object gained some performance improvements as well as a
number of bugfixes.
- Several incompatibilities of wsgi module with the WSGI spec have been fixed.

Read the full changelog entry here: http://www.gevent.org/changelog.html
***********************************************************************
To unsubscribe, send an e-mail to majordomo <at> freehaven.net with
unsubscribe libevent-users    in the body.

headers

Shiqing Fan | 5 Feb 2010 16:30

structure redefinition in _libevent_time.h and winsock2.h


Hi,

When using libevent 1.4.13 in another project on Windows, we found some 
structure are redefined in _libevent_time.h and winsock2.h, for example 
'struct timeval'.

I made a patch to solve this problem, but I'm not sure if it's correct 
to exclude _libevent_time.h on Windows.

Regards,
Shiqing

-- 
--------------------------------------------------------------
Shiqing Fan                          http://www.hlrs.de/people/fan
High Performance Computing           Tel.: +49 711 685 87234
  Center Stuttgart (HLRS)            Fax.: +49 711 685 65832
Address:Allmandring 30               email: fan <at> hlrs.de    
70569 Stuttgart

--- devpoll.orig.c	2010-02-05 15:48:17.375000000 +0100
+++ devpoll.c	2010-02-05 15:50:10.390625000 +0100
 <at>  <at>  -33,7 +33,9  <at>  <at> 
 #ifdef HAVE_SYS_TIME_H
 #include <sys/time.h>
 #else
+#  ifndef HAVE_WINSOCK2_H
 #include <sys/_libevent_time.h>
+#  endif
 #endif
 #include <sys/queue.h>
 #include <sys/devpoll.h>
--- poll.orig.c	2010-02-05 15:49:03.843750000 +0100
+++ poll.c	2010-02-05 15:52:03.281250000 +0100
 <at>  <at>  -34,7 +34,9  <at>  <at> 
 #ifdef HAVE_SYS_TIME_H
 #include <sys/time.h>
 #else
+#  ifndef HAVE_WINSOCK2_H
 #include <sys/_libevent_time.h>
+#  endif
 #endif
 #include <sys/queue.h>
 #include <poll.h>
--- event.orig.c	2010-02-05 15:48:41.250000000 +0100
+++ event.c	2010-02-05 15:54:55.890625000 +0100
 <at>  <at>  -36,8 +36,10  <at>  <at> 
 #include <sys/types.h>
 #ifdef HAVE_SYS_TIME_H
 #include <sys/time.h>
-#else 
+#else
+#  ifndef HAVE_WINSOCK2_H
 #include <sys/_libevent_time.h>
+#  endif
 #endif
 #include <sys/queue.h>
 #include <stdio.h>
--- kqueue.orig.c	2010-02-05 15:48:51.468750000 +0100
+++ kqueue.c	2010-02-05 15:54:53.703125000 +0100
 <at>  <at>  -36,7 +36,9  <at>  <at> 
 #ifdef HAVE_SYS_TIME_H
 #include <sys/time.h>
 #else
+#  ifndef HAVE_WINSOCK2_H
 #include <sys/_libevent_time.h>
+#  endif
 #endif
 #include <sys/queue.h>
 #include <sys/event.h>
--- epoll.orig.c	2010-02-05 15:48:30.343750000 +0100
+++ epoll.c	2010-02-05 15:50:54.546875000 +0100
 <at>  <at>  -34,7 +34,9  <at>  <at> 
 #ifdef HAVE_SYS_TIME_H
 #include <sys/time.h>
 #else
+#  ifndef HAVE_WINSOCK2_H
 #include <sys/_libevent_time.h>
+#  endif
 #endif
 #include <sys/queue.h>
 #include <sys/epoll.h>
--- log.orig.c	2010-02-05 15:48:57.609375000 +0100
+++ log.c	2010-02-05 15:51:47.359375000 +0100
 <at>  <at>  -50,7 +50,9  <at>  <at> 
 #ifdef HAVE_SYS_TIME_H
 #include <sys/time.h>
 #else
+#  ifndef HAVE_WINSOCK2_H
 #include <sys/_libevent_time.h>
+#  endif
 #endif
 #include <stdio.h>
 #include <stdlib.h>
--- select.orig.c	2010-02-05 15:49:10.531250000 +0100
+++ select.c	2010-02-05 15:52:15.281250000 +0100
 <at>  <at>  -34,7 +34,9  <at>  <at> 
 #ifdef HAVE_SYS_TIME_H
 #include <sys/time.h>
 #else
+#  ifndef HAVE_WINSOCK2_H
 #include <sys/_libevent_time.h>
+#  endif
 #endif
 #ifdef HAVE_SYS_SELECT_H
 #include <sys/select.h>
--- configure.orig	2010-02-05 16:15:26.218750000 +0100
+++ configure	2010-02-05 16:15:31.171875000 +0100
 <at>  <at>  -20478,7 +20478,7  <at>  <at> 

 

-for ac_header in fcntl.h stdarg.h inttypes.h stdint.h poll.h signal.h unistd.h sys/epoll.h
sys/time.h sys/queue.h sys/event.h sys/param.h sys/ioctl.h sys/select.h sys/devpoll.h port.h
netinet/in6.h sys/socket.h
+for ac_header in fcntl.h stdarg.h inttypes.h stdint.h poll.h signal.h unistd.h sys/epoll.h
sys/time.h sys/queue.h sys/event.h sys/param.h sys/ioctl.h sys/select.h sys/devpoll.h port.h
netinet/in6.h sys/socket.h winsock2.h
 do
 as_ac_Header=`$as_echo "ac_cv_header_$ac_header" | $as_tr_sh`
 if { as_var=$as_ac_Header; eval "test \"\${$as_var+set}\" = set"; }; then

_______________________________________________
Libevent-users mailing list
Libevent-users <at> monkey.org
http://lists.monkey.org:8080/listinfo/libevent-users

headers

fokenrute | 6 Feb 2010 00:22

evhttp_request_free segfaults

I'm writing a little HTTP server in C with libevent-2.0.3-alpha.

Somewhere in my code I call evhttp_request_own then I put a pointer to the evhttp_request object in an array shared with another thread.

In the other thread, I retrieve the evhttp_request pointer and send a reply with evhttp_send_reply, and when I try to call evhttp_request_free to release memory allocated to the evhttp_request object, it produce a segfault.

Here is the stack trace I get using GDB :

#0 0x00007ffff7bc48b7 in evhttp_clear_headers (headers=0x602ad0) at http.c:1349

#1 0x00007ffff7bc4c3e in evhttp_request_free (req=0x602fb0) at http.c:2814

#2 0x00007ffff7bc6b96 in evhttp_send_done (evcon=0x602af0, arg=<value optimized out>) at http.c:1983

#3 0x00007ffff7bbea48 in bufferevent_writecb (fd=18, event=<value optimized out>, arg=0x602c50) at bufferevent_sock.c:262

#4 0x00007ffff7bb6020 in event_process_active_single_queue (base=0x602030, flags=<value optimized out>) at event.c:925

#5 event_process_active (base=0x602030, flags=<value optimized out>) at event.c:978

#6 event_base_loop (base=0x602030, flags=<value optimized out>) at event.c:1156

#7 0x0000000000401404 in main (argc=1, argv=0x7fffffffe558) at ../src/Frontend.c:184

headers

Niels Provos | 6 Feb 2010 05:28

Re: evhttp_request_free segfaults

On Fri, Feb 5, 2010 at 3:22 PM, fokenrute <fokenrute <at> gmail.com> wrote:
> In the other thread, I retrieve the evhttp_request pointer and send a reply
> with evhttp_send_reply, and when  I try to  call evhttp_request_free to
> release memory allocated to the evhttp_request object, it produce a
> segfault.

evhttp_send_reply will free the request object when it's done sending.
  You should not free it yourself.

Niels.
***********************************************************************
To unsubscribe, send an e-mail to majordomo <at> freehaven.net with
unsubscribe libevent-users    in the body.

25	May 2013
27	April 2013
49	March 2013
46	February 2013
65	January 2013
39	December 2012
40	November 2012
42	October 2012
58	September 2012
58	August 2012
26	July 2012
61	June 2012
46	May 2012
66	April 2012
47	March 2012
58	February 2012
48	January 2012
40	December 2011
35	November 2011
20	October 2011
40	September 2011
88	August 2011
59	July 2011
66	June 2011
78	May 2011
46	April 2011
34	March 2011
62	February 2011
44	January 2011
71	December 2010
110	November 2010
72	October 2010
75	September 2010
50	August 2010
59	July 2010
33	June 2010
30	May 2010
46	April 2010
76	March 2010
38	February 2010
56	January 2010
46	December 2009
44	November 2009
58	October 2009
31	September 2009
17	August 2009
41	July 2009
41	June 2009
59	May 2009
58	April 2009
46	March 2009
32	February 2009
35	January 2009
19	December 2008
19	November 2008
10	October 2008
20	September 2008
25	August 2008
30	July 2008
60	June 2008
46	May 2008
37	April 2008
26	March 2008
15	February 2008

Mon	Tue	Wed	Thu	Fri	Sat	Sun

1	2	3	4	5	6	7
8	9	10	11	12	13	14
15	16	17	18	19	20	21
22	23	24	25	26	27	28