Morand, Guy | 25 Aug 11:41 2015
Picon

epoll fd in libevent

Hallo libevent developers,

I want to monitor GPIO lines under Linux and I'm using libevent. As I understand, for portability issues, it
is not possible to monitor GPIOs with libevent because the "exception" event is not cross platform.

I saw in a thread of the libevent mailing list that it would be possible to create an epoll and register the
file descriptor in libevent: 
http://archives.seul.org/libevent/users/Apr-2012/msg00026.html

I tried this solution but the problem is that I only get the "POLLPRI" event once at startup, despite I
register the epoll fd in libevent with "EV_READ | EV_PERSIST".

Some code talks more than too many explanation, here is how I register the GPIO in epoll:
int EPollEventLoop::addExceptionHandler(libds::IIoHandler& handler)
{
    struct epoll_event event;

    // Init must be called
    assert(m_epollFd >= 0);

    // Add to epoll
    event.data.fd = handler.getIoHandle();
    event.events = EPOLLPRI;    
    if (epoll_ctl(m_epollFd, EPOLL_CTL_ADD, handler.getIoHandle(), &event) != 0)
    {
        ILog_msg(NULL, LOG_LEVEL_ERR, "Cannot add event to epoll %s", strerror(errno));
        return -1;
    }

    return 0;
(Continue reading)

xiaoxin gao | 18 Aug 07:35 2015
Picon

libevent performance problem for detect the socket is readable

Hi all,

 I use libevent in my program, but I meet some performance problem.

I create one class named SocketEventBase. SocketEventBase is one thread and use event_base_dispatch() to check socket.

There is one list<socket*> in the SocketEventBase. Use SocketEventBase to listen all the socket in the list.

When the socket is ready to read, libevent framework will remove socket from the list and I will use another thread(named Handler) to deal the request by using
the callback function.

After Handler complete the request, i will add the socket to event base again.

So during the run-time, there are lots of event will be remove from and add to
the event base.

I try to use more event base to get better performance. I use handler-thread-pool-num to control the num of SocketEventBase. Assume handler-thread-pool-num is four, all requests will be dispatch to the four event base.

When i don't use libevent, one thread will service one socket.

After i use libevent, i wish less threads can service same num of socket and will get better performance.

But i test my program as below.

I use 300 client threads to test my program.
(It is a sysbench oltp test, and libevent is used inside a proxy, which is located between the sysbench and backend mysql)

When i don't use libevent, i get 10000~12000 tps.

After i use libevent, different handler-thread-pool-num get different result.

handler-thread-pool-num          tps
1                               1101.17
2                               2065.75
4                               3354.30
8                               3995.18
16                              4308.92


Then i monitor my program like below:
+----------------------+-----------+--------+
| Monitor_point        | Cost_time | %      |
+----------------------+-----------+--------+
| session_other        | 37462     | 78.36  |
| session_svc          | 47805     | 100.00 |
| session_switch       | 484       | 1.01   |
| session_work         | 9670      | 20.23  |
+----------------------+-----------+-------+

session_svc : all the time of the requests be done
session_switch : the time is when socket is ready to read, move request from SocketEventBase to Handler.
session_work : the time is Handler deal the requests
session_other : the time is from adding socket to SocketEventBase's list to find the socket is ready to read. (The time is also include waiting for sysbench client to send the request, but it still too much, cause the  session_work is only 20%, which also
includes the time for backend mysql handle the sql request and send back result )

I don't know why session_other needs so much time. Is libevent need so much time to find the socket is ready?

some other test result as below:

client thread = 30
handler-thread-pool-num=10
tps 2696.23
session_other 14.56%

client thread = 60
handler-thread-pool-num=10
tps 4323.21
session_other 27.18%

client thread = 90
handler-thread-pool-num=10
tps 5019.76
session_other 41.99%

client thread = 120
handler-thread-pool-num=10
tps 4893.41
session_other 54.31%
Mark Ellzey | 25 Jul 09:49 2015
Picon

Sunset of SourceForge.

September 1st, 2015; we will officially close down the SourceForge repository for libevent. All of the packages and code will remain, but all further development and management will be done on Github. (So grab an account there if you don't have one!)

I've accidentally been neglecting SF as of late -  I always forget to look at my spam folder, I will address these before the move.

The decision was due to many reasons. SourceForge lacks: security, no-advertisements, popups,  not to mention it's incredibly slow. 

Github has become a standard for a reason: intuitive,  easy to use, and transparent about any problems that arise.

Tomorrow I will be migrating the current issues and pull requests  from SourceForge over to Github using some hacked up scripts.
 
So..

Remember, remember, the 1st of September.
Bill Vaughan | 24 Jul 03:56 2015
Picon

leak in be_pair_flush

Hi all,

I added bufferevent pairs to my project a few weeks ago as it was perfect for what I needed.  Got it all working nicely but valgrind was reporting a leak.  I finally tracked down the culprit to the bufferevent pair flush code.

Here is the patch for libevent 2.0.22. The lock was being taken and not released if the mode was BEV_NORMAL.

[bvaughan <at> dev-client tproxy]$ git diff
diff --git a/libevent-2.0.22-stable/bufferevent_pair.c b/libevent-2.0.22-stable/bufferevent_pair.c
index e9ed9f5..2a77dc7 100644
--- a/libevent-2.0.22-stable/bufferevent_pair.c
+++ b/libevent-2.0.22-stable/bufferevent_pair.c
<at> <at> -284,14 +284,15 <at> <at> be_pair_flush(struct bufferevent *bev, short iotype,
 {
        struct bufferevent_pair *bev_p = upcast(bev);
        struct bufferevent *partner;
-       incref_and_lock(bev);
+
        if (!bev_p->partner)
                return -1;
 
-       partner = downcast(bev_p->partner);
+   if (mode == BEV_NORMAL)
+      return 0;
 
-       if (mode == BEV_NORMAL)
-               return 0;
+       incref_and_lock(bev);
+   partner = downcast(bev_p->partner);
 
        if ((iotype & EV_READ) != 0)
                be_pair_transfer(partner, bev, 1);


Thanks,
-Bill

Jesper Lundgren | 19 Jul 12:29 2015
Picon

http client throttle

Hello, I am using libevent http in client mode to send GET requests. Many of the requests I try to send end with http_req_done callbacks where req == NULL and socket error is set to 36 (Operation now in progress). My guess is that the server can not handle the traffic but I am not sure what is the best approach to dynamically adjust the request rate in libevent to prevent socket errors.

Any suggestions?

/JL
Ilya Gordeev | 16 Jul 19:47 2015
Picon

evdns behaviour questions and suggestions

Hello. Using libevent (specifically evdns subsystem) I've encountered 
with some troubles.

1) How should I properly interrupt all pending evdns_getaddrinfo 
requests and free evdns_base? Let's say I've caught signal to interrupt 
working. If I call evdns_getaddrinfo_cancel for all pending evdns 
getaddrinfo requests and immediately call evdns_base_free(evdns_base, 0) 
and event_base_free(base) then some memory leaks occur.
In my testing program I detect memory leaks using mtrace/muntrace. (If 
it's needed I can attach testing program). In one mtrace log I've got this:
=====
Memory not freed:
-----------------
            Address     Size     Caller
0x00000000006035e0     0x30  at 0x7ffff799224f
0x0000000000603620    0x108  at 0x7ffff7992687
0x0000000000603730    0x108  at 0x7ffff7992687
0x0000000000603840     0x30  at 0x7ffff799224f
0x0000000000603880     0x30  at 0x7ffff799238f
0x0000000000603d60     0x30  at 0x7ffff799224f
0x0000000000603e60     0x30  at 0x7ffff799238f
0x0000000000603ea0     0x30  at 0x7ffff799238f
0x0000000000603fc0     0x30  at 0x7ffff799238f
0x0000000000604010    0x108  at 0x7ffff7992687
0x0000000000604120     0x40  at 0x7ffff7bc7496
0x0000000000604170     0x40  at 0x7ffff7bc7496
0x00000000006043e0    0x258  at 0x7ffff798d553
0x0000000000604640    0x108  at 0x7ffff7992687
0x0000000000604900    0x258  at 0x7ffff798d553
0x0000000000604b60    0x258  at 0x7ffff798d553
0x0000000000604f20    0x258  at 0x7ffff798d553
0x0000000000605180    0x258  at 0x7ffff798d553
0x00000000006056a0    0x258  at 0x7ffff798d553
0x0000000000605900    0x258  at 0x7ffff798d553
=====
Then in gdb I've got which code they are:
=====
(gdb) info line *0x7ffff798d553
Line 834 of "libevent-2.1.5-beta/evdns.c" starts at address 
0x7ffff798d553 <reply_schedule_callback+35>
    and ends at 0x7ffff798d556 <reply_schedule_callback+38>.
(gdb) info line *0x7ffff799224f
Line 2861 of "libevent-2.1.5-beta/evdns.c"
    starts at address 0x7ffff799224f <evdns_base_resolve_ipv4+63> and 
ends at 0x7ffff7992252 <evdns_base_resolve_ipv4+66>.
(gdb) info line *0x7ffff799238f
Line 2900 of "libevent-2.1.5-beta/evdns.c"
    starts at address 0x7ffff799238f <evdns_base_resolve_ipv6+63> and 
ends at 0x7ffff7992392 <evdns_base_resolve_ipv6+66>.
(gdb) info line *0x7ffff7992687
Line 4650 of "libevent-2.1.5-beta/evdns.c" starts at address 
0x7ffff7992687 <evdns_getaddrinfo+551>
    and ends at 0x7ffff799268a <evdns_getaddrinfo+554>.
(gdb) info line *0x7ffff7bc7496
Line 824 of "libevent-2.1.5-beta/evutil.c" starts at address 
0x7ffff7bc7496 <evutil_new_addrinfo_+70>
    and ends at 0x7ffff7bc7499 <evutil_new_addrinfo_+73>.
=====

2) It would be very useful in some cases if there was additional evdns 
functions similar to some event_* such as: event_base_foreach_event(), 
event_get_callback() and event_get_callback_arg().

3) If evdns_getaddrinfo_cancel is called then evdns_getaddrinfo_request 
callback will be called with result = EVUTIL_EAI_CANCEL (not 
DNS_ERR_CANCEL). Am I right? If yes it was not easy to figure out, in 
documentation it is not very clear.

***********************************************************************
To unsubscribe, send an e-mail to majordomo <at> freehaven.net with
unsubscribe libevent-users    in the body.

Mark Ellzey | 7 Jul 22:54 2015
Picon

test

sorry for the spam. testing.
Kaustubh Deorukhkar | 23 Jun 12:34 2015
Picon

libevent HTTP support feature list

Hello,

I am new to libevent and would like to know how is the support for HTTP protocol. I am evaluating it for a http server and would help if you all can share experience with using libevent HTTP in prod.

- Does it support full HTTP/1.1 ?
- HTTP pipe-lining (found some posts about issues with pipe-lining on keep alive, is it fixed and stable?)
- Expect 100 support for managing requests
- Do we have a feature list of what all is supported?

Will be very helpful if someone can share or point me to docs if any?

Thanks in advance,
Kaustubh
Goetz T. Fischer | 14 Jun 17:57 2015
Picon

echo server, close after reply

hi,

this is probably trivial but i couldn't find a way of getting it done.

using the always popular echo server example from the docs i just want the 
connection to close after sending the "echo". BEV_OPT_CLOSE_ON_FREE is set 
already so i'd just add bufferevent_free(bev) after evbuffer_add_buffer(output, 
input). however as you probably have guessed already no reply is sent then at 
all.
i don't seem to be the only one wondering about this. i found two other guys 
asking the same here

http://www.codedisqus.com/CzVjVXqXkX/how-to-close-socket-after-writing-in-libevent-with-bufferevents.html 
and here

http://stackoverflow.com/questions/15995659/libevent-writes-to-the-socket-only-after-second-buffer-write. 
both however without a reply.

so my simple question is: how can i send the reply and then close the 
connection?

any hint is welcome and thanks in advance!

--
R-A-C
Götz T. Fischer CertIT&Comp
+49(0)7225/98 98 79
g.fischer <at> r-a-c.de
r-a-c.de
***********************************************************************
To unsubscribe, send an e-mail to majordomo <at> freehaven.net with
unsubscribe libevent-users    in the body.

Tomer Heber | 5 Jun 22:46 2015

Infinite Loop in libevent.

Hi,


I've been using libevent (2.0.21) for some time now. I have encountered a very rare and strange issue on one of our production environments.

The machine was at 100% cpu, and had been caught on some kind of infinite loop. Using GDB I traced it to libevent.


Just to clarify, I verified with tcpdump that the machine was not under a DDOS attack.


strace showed a constant call to "gettimeofday" (or one of it's equivalent), and a failure due to too many open file descriptors.


I was able to obtain the following from gdb

#0  0x00007fd495cbf2d3 in vfprintf () from /lib64/libc.so.6

#1  0x00007fd495d7aaa0 in __vsnprintf_chk () from /lib64/libc.so.6

#2  0x00007fd4969d71c2 in vsnprintf (buf=0x7fd48abfc468 ": Too many open files", buflen=1000, format=<value optimized out>, ap=<value optimized out>) at /usr/include/bits/stdio2.h:78

#3  evutil_vsnprintf (buf=0x7fd48abfc468 ": Too many open files", buflen=1000, format=<value optimized out>, ap=<value optimized out>) at evutil.c:1577

#4  0x00007fd4969d7263 in evutil_snprintf (buf=<value optimized out>, buflen=<value optimized out>, format=<value optimized out>) at evutil.c:1554

#5  0x00007fd4969d623d in _warn_helper (severity=2, errstr=0x7fd495dd0d70 "Too many open files", fmt=<value optimized out>, ap=<value optimized out>) at log.c:183

#6  0x00007fd4969d6634 in event_sock_warn (sock=<value optimized out>, fmt=0x7fd4969ee50b "Error from accept() call") at log.c:124

#7  0x00007fd4969d365c in listener_read_cb (fd=41, what=<value optimized out>, p=0x7fd460004dc0) at listener.c:441

#8  0x00007fd4969c8f8c in event_process_active_single_queue (base=0x1046480, flags=0) at event.c:1350

#9  event_process_active (base=0x1046480, flags=0) at event.c:1420

#10 event_base_loop (base=0x1046480, flags=0) at event.c:1621


Since it was a production environment there was little more I could in do. In the end GDB crashed the process (or the OS?).


Any ideas?


10x,

Tomer.


René Berber | 31 May 22:03 2015

Source packages weirdness; probably a real problem

Hi,

Something weird is going on with libevent source packages:

* Their gpg signatures check, they have always check.
* I'm using MXE which also verifies a (sha1) checksum:
 + Version 2.0.22, downloaded about a week ago, had the checksum of what
2.0.21 has now... that shouldn't happen.
 + Version 2.0.21 currently has a different checksum to what it had
originally, about a year ago... that also shouldn't happen.
* Using those versions results in part of my app not working, the part
that uses libevent to set up a http/RPC server... this proves that at
least version 2.0.21 is not what it was originally (i.e. I've used it
many times over the last year to build my app, and it worked fine).

Any ideas?

I haven't tried to find what the problem is, but from my point of view,
it looks like a security break at your end, and a result that I cannot
trust on my end (not only that it doesn't work, but I can't be sure is
not doing something else).
--

-- 
René Berber

Attachment (smime.p7s): application/pkcs7-signature, 5069 bytes

Gmane