Fongang Dassi Jean | 27 Nov 02:29 2014
Picon

Best way for additional events registration in the same thread

Hello,


"Ordinarily, the loop will exit as soon as it has no pending or active events. You can override this behavior by passing the EVLOOP_NO_EXIT_ON_EMPTY flag---for example, if you’re going to be adding events from some other thread."

This seems to imply that it is only recommended to add events if another thread is used?

This program: https://raw.githubusercontent.com/bagder/curl/master/docs/examples/hiperfifo.c manages to register additional events after dispatch is called, but seems to rely on the existence of a persisting event.

Is that the only way? What is the best way?

Thanks in advance
svante karlsson | 20 Nov 10:44 2014
Picon

mixing libevent and boost::asio

Is there a good way of interacting between those two libraries?

I'm trying to mix two single threaded libraries - one built for each platform. It's simple to poll each library but this this seems to mean either introducing a sleep in the loop of spinning a full speed.

a naive approach...
....
  while (true)
    {
       // does not block but might handle several events
       int s1 = event_base_loop(_event_base, EVLOOP_NONBLOCK); 
      
       // handles at most 1 event
      size_t n = ios.poll();  
       if (n>0)
           continue;
       boost::this_thread::sleep(boost::posix_time::milliseconds(1)); 
    }

Is there a way of waiting for both sets of file descriptors so I don't have to rely on the timer or a busy loop to poll both events?

I know I can have one thread running the libevent queue and another the boost::asio eventloop but that gives problems when the callbacks in each subsystem calls the other.

I'm familiar with boost but not libevent so please bear with me if this question is trivial in libevent context...

/svante


jason | 20 Nov 08:06 2014

a bug of libevent(2.1.4-alpha) IOCP: socket is closed twice when bufferevent is freed

I ran into a strange situation where some new connections were randomly closed just after many client connections were closed. I finally get the reason after diving into source code and debugging for several hours.

 

When bufferevent is freed, the internal socket is closed at be_async_ctrl().

Call stack:

>       51CY.exe!be_async_ctrl(bufferevent * bev, bufferevent_ctrl_op op, bufferevent_ctrl_data * data)  Line 673  C

       51CY.exe!bufferevent_cancel_all_(bufferevent * bev)  Line 884 + 0x15 bytes     C

       51CY.exe!bufferevent_free(bufferevent * bufev)  Line 786 + 0x9 bytes       C

       51CY.exe!be_filter_unlink(bufferevent * bev)  Line 234 + 0xf bytes     C

       51CY.exe!bufferevent_decref_and_unlock_(bufferevent * bufev)  Line 696 + 0xf bytes      C

       51CY.exe!bufferevent_free(bufferevent * bufev)  Line 787 + 0x9 bytes       C

 

Bufferevent_free() also forwards a event callback to event_base.

Call stack:

>       51CY.exe!event_callback_finalize_many_(event_base * base, int n_cbs, event_callback * * evcbs, void (event_callback *, void *)* cb)  Line 2210   C

       51CY.exe!bufferevent_decref_and_unlock_(bufferevent * bufev)  Line 713 + 0x18 bytes   C

       51CY.exe!bufferevent_free(bufferevent * bufev)  Line 787 + 0x9 bytes       C

 

The same socket is secondly closed when the event callback is invoked in the next event_base loop.

Callstack:

>       51CYClient.exe!evutil_closesocket(int sock)  Line 425 C

       51CYClient.exe!be_async_destruct(bufferevent * bev)  Line 386 + 0x9 bytes     C

       51CYClient.exe!bufferevent_finalize_cb_(event_callback * evcb, void * arg_)  Line 734 + 0xf bytes C

       51CYClient.exe!event_process_active_single_queue(event_base * base, evcallback_list * activeq, int max_to_process, const timeval * endtime)  Line 1604 + 0xe bytes  C

       51CYClient.exe!event_process_active(event_base * base)  Line 1668 + 0x14 bytes  C

       51CYClient.exe!event_base_loop(event_base * base, int flags)  Line 1890 + 0x9 bytes     C

 

So socket of new connection will be closed by this mean if the new socket handle value equals to the old one.

Ansuman Mohanty | 18 Nov 11:49 2014
Picon

lib event crash in event_assign()

Hi,

I am seeing a crash in the event_assign() function.
Can someone please tell me what I am doing wrong?
What should I do?

Thanks,
Ansuman
jason | 14 Nov 10:04 2014

a bug in libevent IOCP

These days, I use libevent IOCP for my server programming, but I was trapped in a situation where some network package data sent by client first time will be lost. The following codes suffers from that bug.

 

                   m_pBufferEvent = bufferevent_socket_new( pBase, fSocket, BUFFER_EVENT_OPTION );

                   bufferevent_enable( m_pBufferEvent, EV_WRITE | EV_READ );

bufferevent_setcb( m_pBufferEvent, OnRead, NULL, OnEvent, &m_qwSessionID );

 

After some days diving into libevent source code and debugging, I find the reason. Bufferevent_enable() will invoke user callback if there is a data in input buffer, but now user callback hasn’t been set. So you have to set user callback before you enable buffer read and write. Like this:

 

m_pBufferEvent = bufferevent_socket_new( pBase, fSocket, BUFFER_EVENT_OPTION );

bufferevent_setcb( m_pBufferEvent, OnRead, NULL, OnEvent, &m_qwSessionID );

                   bufferevent_enable( m_pBufferEvent, EV_WRITE | EV_READ );

 

I think bufferevent_setcb() should call user callback when there is some data. Or you must add relevant comments to this function to warn developers.

Mark Ellzey | 12 Nov 19:03 2014
Picon

Re: evtimer_new "leak"

On Mon, Oct 06, 2014 at 04:50:13PM -0700, Michael Herf wrote:
> I wrote some code that was leaking on every call to evtimer_new -- I
> was definitely calling evtimer_del afterwards (sometimes before the
> timer fired).
> 
> To fix the leak, I found out I actually needed to call event_free,
> instead. Took me a while to go read the libevent code and figure
> this out.
> 
> This seems kinda confusing API-wise, and if my understanding of it
> is correct, adding an "evtimer_free" macro (also, evsignal_free,
> ...) or a better explanation could save people some time.
> 

+1 on having evtimer_free(). Though the best explanation is the
following:

"As a convenience, there are a set of macros beginning with evtimer_
that you can use in place of the event_* calls to allocate and
manipulate pure-timeout events. Using these macros provides no benefit
beyond improving the clarity of your code."

Note where it states it provides no benefit beyond the readability of
your code.

Casually scanning your code, what pops out more as a timer than any
other type of event? 

This?

  struct event * ev = event_new(base, -1, 0, callback, NULL);
  struct timeval tv = { 1, 0 };

  event_add(ev, &tv);

Or this?

   struct event * ev = evtimer_new(base, callback, NULL);
   struct timeval tv = { 1, 0 };

   evtimer_add(ev, &tv);

***********************************************************************
To unsubscribe, send an e-mail to majordomo <at> freehaven.net with
unsubscribe libevent-users    in the body.

Mark Ellzey | 12 Nov 18:46 2014
Picon

Re: bufferevent_openssl's output callback

On Mon, Oct 06, 2014 at 10:17:43AM -0700, Catalin Patulea wrote:
> I'm trying to get evhttp+bufferevent_openssl working without
> BEV_OPT_DEFER_CALLBACKS because I suspect it's the cause of some
> crashes on connection close edge cases.
> 

You've just brought back some horrible memories of getting bevossl
working non deferred. And you are correct, there are so many edge cases
it hurt my brain. One patch exposed a new bug which needed another
patch, and then another. It was a nightmare. (worst week of my life).

Anyway,

Are you by chance running in a threaded manner?

***********************************************************************
To unsubscribe, send an e-mail to majordomo <at> freehaven.net with
unsubscribe libevent-users    in the body.

Krishna Sankaran | 11 Nov 02:11 2014
Picon

Re: evdns resolve functions and /etc/hosts


Hi,

It appears that evdns_addrinfo functions doesn't return any static values configured in /etc/hosts

    struct evutil_addrinfo hints;
    ZERO_STRUCT(hints);
    hints.ai_family = AF_INET;
    hints.ai_socktype = SOCK_STREAM;

    req = evdns_getaddrinfo(a_dns,
                                         "www.zyx.com",
                                         NULL,
                                         &amp ;hints,
                                         dns_cb,
                                          ip_addr);

Is there any option to make it look at /etc/hosts as well.

Thanks a lot for the help,
Krishna


On Wednesday, November 5, 2014 5:07 PM, Krishna Sankaran <krishnasankaran <at> yahoo.com> wrote:


Hi,

It appears that evdns_addrinfo functions doesn't return any static values configured in /etc/hosts

    struct evutil_addrinfo hints;
   ZERO_STRUCT(hints);
    hints.ai_family = AF_INET;
    hints.ai_socktype = SOCK_STREAM;

    req = evdns_getaddrinfo(a_dns,
                                         "www.zyx.com",
                                         NULL,
                                         &hints,
                                         dns_cb,
                                          ip_addr);

Is there any option to make it look at /etc/hosts as well.

Thanks a lot for the help,
Krishna


Marcin Szewczyk | 31 Oct 14:52 2014

evbuffer_add_file() file descriptor exhaustion experiment

Hi,

I am struggling with exhaustion of file descriptors when using
evbuffer_add_file(). My application responds to every line of input with
contents of a file. I don't know how to efficiently control open
descriptors number so when input rate is high I hit the open file
descriptors limit very quickly. I would like to ask how to resolve the
problem correctly. I have seen that long time ago someone has asked a
similar question[1], but there was no answer.

I attach a short source code to present my problem.

Ideas made up so far (excerpt from the attached source):
    Bad ideas:
    1. Depending on bytes count in the output evbuffer would be 
       troublesome if the sent file changes between 
       evbuffer_add_file() calls -- it would require a list of last 
       N file sizes to check if evbuffer size is below the sum of N-K 
       sizes (N - high water, N-K - low water).
    2. Checking fd number returned from open() wouldn't be enough
       because after sending contents associated with the oldest call 
       to evbuffer_add_file() the fd number is freed and the next 
       call to open() returns a low number even though descriptors 
       with higher numbers are still in use.
    3. Iterating through file descriptors to check their states seems 
       inefficient.
    4. Listing files from /proc/self/fd seems inefficient.
    5. One could implement a rate limit on requests but it's a quirk.  
       There is no guarantee of the output rate.
    6. ???

    Possibly useful ideas:
    1. extend evbuffer_cb_info to have open file descriptors count
    2. implement evbuffer descriptor closed callback
    3. implement a function to check evbuffer open descriptors count
    4. ???

[1] http://archives.seul.org/libevent/users/Nov-2011/msg00019.html

--

-- 
Marcin Szewczyk                       http://wodny.org
mailto:Marcin.Szewczyk <at> wodny.borg  <- remove b / usuń b
xmpp:wodny <at> ubuntu.pl                  xmpp:wodny <at> jabster.pl
Attachment (evbuffer_add_file.c): text/x-csrc, 5263 bytes
nick | 13 Oct 10:14 2014

Disabling callbacks and closing - block until callbacks finished

Situation:
----------

I am building a multithreaded socket server built with Libevent.   The  
event loop is run in its own thread and each callback creates a task  
into a queue.   A pool of worker threads then processes the tasks, (of  
course ensuring that tasks for any given connection are processed in  
the correct order).

When a close event arrives, a task is created to close the connection  
and free all related structures.   When this task is processed by a  
worker thread, it does the following things:

1. Disable all callbacks
2. Close the connection
3. Remove all pending tasks in the task queue for this connection

Problem:
--------

The problem is that after the connection is closed, there could still  
be events in the Libevent event loop either waiting to be processed in  
a callback or actually being processed by the event thread (i.e.  
callback running).   This could lead to tasks being created for the  
connection _after_ step 3 which would result in tasks in the task  
queue for a closed connection - which I want to avoid.

Solution?
---------

I could use a mutex to lock the connection in the event callbacks, but  
I do not want to introduce blocking methods in the event loop thread.

I could ensure that a connection is valid when I take a task.

But ideally I would like to the bufferevent close method, or the  
callback disable methods to block until all curently running callbacks  
for that bufferevent have finished.   This way I could be sure that no  
tasks can be created after the connection is closed.

Question:
---------

Do the methods work like this, or is there a way to achieve this?

Many thanks in advance.

Nick.

***********************************************************************
To unsubscribe, send an e-mail to majordomo <at> freehaven.net with
unsubscribe libevent-users    in the body.

gerkenjohannes | 19 Sep 14:40 2014
Picon

Re: Re: Assert fail in evhttp_connection_fail_ with req=0x00 on error=EVREQ_HTTP_TIMEOUT at http.c:691

I'm using the "master" from github.

The problem only occurs, when I set a timeout/retry for an https connection(to an server with port 443
closed). The segfault/assert-fail occurs, if I use EVENT_BASE_FLAG_EPOLL_USE_CHANGELIST in
combination with the timeouts/retrys.

Testcase:
add the following to sample/https-client.c

struct timeval retry_timeout;
retry_timeout.tv_sec=5;
retry_timeout.tv_usec=0;
evhttp_connection_set_timeout(evcon,15);
evhttp_connection_set_retries(evcon,3);
evhttp_connection_set_initial_retry_tv(evcon,&retry_timeout);

execute: https_client -url https://libevent.org

output:

[warn] Epoll ADD(1) on fd 6 failed.  Old events were 0; read change was 1 (add); write change was 0 (none);
close change was 0 (none): Bad file descriptor
[debug] event_add: event: 0x25a5b30 (fd 6),  EV_WRITE  EV_TIMEOUT call 0x7f1942897ae4
[warn] Epoll ADD(4) on fd 6 failed.  Old events were 0; read change was 0 (none); write change was 1 (add);
close change was 0 (none): Bad file descriptor
[debug] event_process_active: event_callback 0x25a5c40, closure 3, call 0x7f1941fc8879
some request failed - no idea which one though!
socket error = Bad file descriptor (9)
 

Gesendet: Donnerstag, 04. September 2014 um 09:44 Uhr
Von: "Azat Khuzhin" <a3at.mail <at> gmail.com>
An: libevent-users <at> freehaven.net
Cc: libevent-users <at> seul.org
Betreff: Re: [Libevent-users] Assert fail in evhttp_connection_fail_ with req=0x00 on
error=EVREQ_HTTP_TIMEOUT at http.c:691
On Fri, Aug 29, 2014 at 06:30:26PM +0200, gerkenjohannes <at> web.de wrote:
> Hi,
>
> when I add new evhttp requests to an running event_loop, I get "epoll ADD: Bad file descriptor" and later "SIGABRT".
> The event_base is running in an seperate worker thread, the main thread is doing the evhttp_make_request.
>
> Output from GDB:
> http://paste.ubuntu.com/8176676/[http://paste.ubuntu.com/8176676/]
>
> Output from GDB with libevent debugging on:
> http://paste.ubuntu.com/8176681/[http://paste.ubuntu.com/8176681/]
>
> I get the assert fail in the folowing 2 cases:
>
> case 1:
> 1. add new requests to the running event_base
>
> case 2:
> 1. call loop_brake from the main thread
> 2. wait for the loop to "brake" in the worker thread
> 3. put the worker thread to sleep
> 4. add new requests
> 5. wake up the worker thread
> 6. run the event_loop
>
> Is this bug in libevent or my code?

Which version of libevent do you have?
Could you produce simple program or regress test for http (inside
libevent), to test it?

>
> Best regards,
> ***********************************************************************
> To unsubscribe, send an e-mail to majordomo <at> freehaven.net with
> unsubscribe libevent-users in the body.

--
Respectfully
Azat Khuzhin
***********************************************************************
To unsubscribe, send an e-mail to majordomo <at> freehaven.net with
unsubscribe libevent-users in the body.
***********************************************************************
To unsubscribe, send an e-mail to majordomo <at> freehaven.net with
unsubscribe libevent-users    in the body.


Gmane