Rick van Rein | 14 Feb 15:47 2015
Picon

Fwd: Driving a thread pool from libev

Hi,

> Excellent solution, it indeed avoid memcpy().

Thanks :-D

Conceptually the most logical model would be if libev would run multiple callbacks, each running in
workers from the thread pool; and if it would use this mechanism to avoid starting multiple such callbacks
on a single watcher.

> When you say "yeild()", it's some notification like ev_async_send(), right?

Not quite.  My thinking was that a main thread could spark off multiple callback threads, but these may not
have run when the main thread loops back for another poll() invocation.  This would be sure to give all
sparked-off threads an opportunity to remove the revents from the watcher.  Only threads that don’t act
on their watcher's revents in time would be temporarily removed from the poll() list.  When the threaded
callback finished, they would of course be added back in the poll() list.

The ideal is to run this pthread_yield() once per thread, just before returning to poll() — calling it
from every callback function is possible, but wasteful as a single round over all threads is sufficient
for each invocation of poll().  I don’t know yet if there are macro’s I can define to do that.

I’m thinking even further… the main thread could simply be one from the thread pool.  It would start off
other slaves, until there’s more work then it has slaves for, in which case it would degrade itself to a
slave.  Finishing slaves would take out one further event in turn, until the last is consumed; by that time,
the slave promotes itself to be the new master, and returns to poll().  This makes threads continue to run in
many cases, which means that the opportunity for lock-free or spinlock-based control is high.  Only when a
thread really needs to wait would it be beneficial to wait for locks.  Sort of the “sleeping barber”
setup, where clients volunteer as barbers :)

(Continue reading)

Rick van Rein | 14 Feb 10:10 2015
Picon

Driving a thread pool from libev

Hello,

I am trying to drive a thread pool from libev, but the design appears to be impossible; or has anyone no this
list done this before?

The application: a TLS Pool [1], with worker threads that read(), then encrypt or decrypt, then write(). 
The workers should be part of a thread pool.  And, assuming that read() is a bit costly [2], I am trying to do
even that operation in the worker threads.

	[1] http://tlspool.arpa2.net
	[2] when the data is not buffered, and also because data would flow through an intermediate buffer causing
an extra copy operation

When work is offloaded to the thread pool, the callback from libev can return and would hopefully cause
other work to be queued, thus keeping the thread pool occupied with encryption/decryption.  The problem
is, I am not sure if libev will avoid triggering an object that has already been queued, since poll() and
friends will return a read opportunity over and over again.  And AFAIK I cannot tell libev to temporarily
stop calling back on a watcher, except by taking it out completely — which I presume is an expensive operation.

The current design uses a thread for each TLS connection, which is wasteful of memory, and this is why I am
looking to create a thread pool with a limited number of shared workers.  I could decide to do the read() in
the callback, if need be, but have not quite given up on a more efficient approach.  Has anyone on this list
seen such a setup with libev?

Thanks,

Rick van Rein
ARPA2.net

_______________________________________________
(Continue reading)

Dmitry Marakasov | 4 Feb 21:33 2015
Picon

Incorrect C11 / cannot build with clang 3.6

Hi!

I'm testing new version of clang (3.6) on various software, and
libev doesn't build with it:

---
libtool: compile:  clang36 -DHAVE_CONFIG_H -I. -O2 -pipe -march=nocona -fstack-protector
-fno-strict-aliasing -MT ev.lo -MD -MP -MF .deps/ev.Tpo -c ev.c  -fPIC -DPIC -o .libs/ev.o
ev.c:477:48: warning: '/*' within block comment [-Wcomment]
/*#define MIN_INTERVAL  0.00000095367431640625 /* 1/2**20, good till 2200 */
                                               ^
ev.c:970:42: error: '_Noreturn' keyword must precede function declarator
  ecb_inline void ecb_unreachable (void) ecb_noreturn;
                                         ^~~~~~~~~~~~
  _Noreturn 
ev.c:773:26: note: expanded from macro 'ecb_noreturn'
  #define ecb_noreturn   _Noreturn
---

Seems like C11 is now the default, however libev code is incorrect
C11-wise, as ecb_noreturn which expands to _Noreturn follows function
definition, while _Noreturn should preceed it.

Depending on whether it's allowed to place __attribute__ before
function, either ecb_noreturn should be moved before ecb_unreachable
declaration, or it should be split to e.g. ecb_pre_noreturn and
ecb_post_noreturn to work for both cases.

Full log with clang 3.4:
https://people.freebsd.org/~amdmi3/libev-clang34.log
(Continue reading)

Dinesh RTP | 22 Jan 06:22 2015
Picon

1/22/2015 5:22:31 AM




http://www.stainlesssteelfabricator.co.uk/tcnxvdmo/cbgdxoljksdqt.umdrdnwxlfgtuovrjkkjshsxyricq




















Dinesh RTP
1/22/2015 5:22:31 AM
_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Hayk Martirosyan | 20 Jan 02:33 2015
Picon

while(true) { ev_run(loop, EVRUN_NOWAIT); } much faster than ev_run(loop);

I'm using libev in a high-performance application, and I've noticed against my expectation that looping ev_run with EVRUN_NOWAIT handles events 5x faster than calling ev_run without the loop. Is this expected behavior?

Best,
Hayk
_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
biliucheng@diditaxi.com.cn | 12 Jan 09:10 2015
Picon

a question when using libev

hi
    I encount a problem when using libev. In a callback function, I want to modify the fd's events.
    I try to use ev_io_set and ev_io_start, but it doesnot work. 
    From libev's code, I guess backend_modify can do it, but it is not an outter interface.
    Could you tell me what  I should do?

Thank you 

biliucheng <at> diditaxi.com.cn
_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Alexander Klauer | 8 Jan 10:45 2015
Picon

ev_io events doc clarification

Hi,

I'd like to suggest the following clarification to the docs:

cvs diff: Diffing .
Index: ev.pod
===================================================================
RCS file: /schmorpforge/libev/ev.pod,v
retrieving revision 1.434
diff -u -r1.434 ev.pod
--- ev.pod	6 May 2014 13:24:39 -0000	1.434
+++ ev.pod	8 Jan 2015 09:34:56 -0000
 <at>  <at>  -1749,7 +1749,9  <at>  <at> 

 =item int events [read-only]

-The events being watched.
+The events being watched. May contain additional (internal) event flags
+besides C<EV_READ> and C<EV_WRITE>, so make sure you use bitmasks when
+you check this member.

 =back

Best regards,
Alexander
Kirill Timofeev | 15 Dec 01:11 2014
Picon

failures with "libev: ev_io_start called with corrupted watcher", ((WL)w)->next != (WL)w)

Hi folks,

I have application using libev, which runs for months without issues. 
But sometime it fails with following message in the log:

Dec 14 03:55:48 els-abacus-prod-01 statsd-router: 2014-12-14 03:55:47 
ERROR ds_health_check_timer_cb: before ev_io_init else branch
Dec 14 03:55:48 els-abacus-prod-01 statsd-router: 2014-12-14 03:55:47 
ERROR ds_health_check_timer_cb: before ev_io_init else branch
Dec 14 03:55:48 els-abacus-prod-01 statsd-router: 2014-12-14 03:55:47 
ERROR ds_health_check_timer_cb: before ev_io_start
Dec 14 03:55:48 els-abacus-prod-01 statsd-router: 2014-12-14 03:55:47 
ERROR ds_health_check_timer_cb: before ev_io_start
Dec 14 03:55:48 els-abacus-prod-01 statsd-router: 2014-12-14 03:55:47 
ERROR ds_health_check_timer_cb: before ev_io_init else branch
Dec 14 03:55:48 els-abacus-prod-01 statsd-router: statsd-router: 
ev.c:3552: ev_io_start: Assertion `("libev: ev_io_start called with 
corrupted watcher", ((WL)w)->next != (WL)w)' failed.

Here is function, where failure occurs:

void ds_health_check_timer_cb(struct ev_loop *loop, struct ev_periodic 
*p, int revents) {
     int i;
     int health_fd;
     struct ev_io *watcher;

     for (i = 0; i < global.downstream_num; i++) {
         watcher = (struct ev_io *)(&global.downstream[i].health_watcher);
         health_fd = watcher->fd;
         if (health_fd < 0) {
             health_fd = socket(AF_INET, SOCK_STREAM, 0);
             if (health_fd == -1) {
                 log_msg(ERROR, "%s: socket() failed %s", __func__, 
strerror(errno));
                 continue;
             }
             if (setnonblock(health_fd) == -1) {
                 close(health_fd);
                 log_msg(ERROR, "%s: setnonblock() failed %s", __func__, 
strerror(errno));
                 continue;
             }
             if (connect(health_fd, (struct sockaddr 
*)(&global.downstream[i].sa_in_health), 
sizeof(global.downstream[i].sa_in_health)) == -1 && errno == EINPROGRESS) {
                 log_msg(ERROR, "%s: before ev_io_init if branch", 
__func__);
                 ev_io_init(watcher, ds_health_connect_cb, health_fd, 
EV_WRITE);
             } else {
                 log_msg(ERROR, "%s: connect() failed %s", __func__, 
strerror(errno));
                 close(health_fd);
                 continue;
             }
         } else {
             log_msg(ERROR, "%s: before ev_io_init else branch", __func__);
             ev_io_init(watcher, ds_health_send_cb, health_fd, EV_WRITE);
         }
         log_msg(ERROR, "%s: before ev_io_start", __func__);
         ev_io_start(loop, watcher);
     }
}

This failure happens during network outages like switch failure. Should 
I do call connect() in else branch and check it for EISCONN? Please let 
me know if there is any good practice for situation like this.

Thanks,
Kirill
Matthew Iversen | 11 Dec 06:21 2014

config.guess and config.sub out of date

Hi there.

I'd like to simply report that the config.guess and config.sub files are very out of date in the libev releases. e.g in 4.19 they are from 2008.

Recently this has caused a user a problem in a python library (gevent) using libev, here: https://github.com/gevent/gevent/issues/502

The most recent versions of these files are from this / last month. Available here:

http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess;hb=HEAD
http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.sub;hb=HEAD

If there's no reason not to it would be great if you could update these for the next release of libev. Many thanks,

Matt Iversen
-- Matt Iversen // matt <at> notevencode.com PGP: 0xc046e8a874522973 // 2F04 3DCC D6E6 D5AC D262 2E0B C046 E8A8 7452 2973
_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Alejandro Mery | 11 Dec 18:16 2014
Picon

[PATCH] configure.ac: fix syntax error in ./configure

dnl eats the new line between both statements

./configure: line 2815: syntax error near unexpected token `ac_config_headers="$ac_config_headers config.h"'
./configure: line 2815: `fi ac_config_headers="$ac_config_headers config.h"'

this is still a problem with autoconf 2.69 compiling libev-4.19 from sources

autoconf 2.69 on Ubuntu 14.04.1 LTS x86_64

thanks,
Alejandro Mery

---
 configure.ac | 1 +
 1 file changed, 1 insertion(+)

diff --git a/configure.ac b/configure.ac
index 7e2d5e3..69e0c43 100644
--- a/configure.ac
+++ b/configure.ac
 <at>  <at>  -5,6 +5,7  <at>  <at>  orig_CFLAGS="$CFLAGS"
 AC_CONFIG_SRCDIR([ev_epoll.c])

 AM_INIT_AUTOMAKE(libev,4.19) dnl also update ev.h!
+
 AC_CONFIG_HEADERS([config.h])
 AM_MAINTAINER_MODE

--

-- 
2.2.0
Noam Camus | 2 Dec 12:33 2014

EV_CHILD_ENABLE/EV_SIGNAL_ENABLE

Hello

 

I saw in "Changes" file that under '4.00 Mon Oct 25 12:32:12 CEST 2010' there is:

-          new EV_CHILD_ENABLE and EV_SIGNAL_ENABLE configurable settings.

 

I wish to know how one should control these definitions?

 

This is since I see no support for that in configure.

Is there a way to use configure to pass extra cflags?

 

Thanks

Noam

_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev

Gmane