Re: strange exit with hand-made socket and bufferevent

On Sun, Feb 21, 2010 at 10:08 AM, Mihai Draghicioiu
<mihai.draghicioiu <at> gmail.com> wrote:
> Thanks Nick!
>
> Yes, the last commit fixed the example. I was compiling it with g++
> and that's probably why it worked.
>
> As for my program, i fixed it by using
> bufferevent_socket_connect_hostname instead of evdns_* calls.
>
> Questions:
> 1. What happens if the dns resolver fails? Does it retry? Does it give
> up? How do I know what happened?

It gives up, I believe.  Check out the "timeout" option described at
http://www.wangafu.net/~nickm/libevent-book/Ref9_dns.html for more
info on how long it waits for an answer by default.

If you're doing the query via bufferevent_socket_connect_hostname()
and the name lookup fails for any reason, you should get an error
callback.  Right now there isn't a good way to extract the DNS error
from it, though I'd love to add one.

> 2. What happened to EVUTIL_CHECK_FMT? It was useful. What can I replace it with.

It got removed, since (as a non-network-related thing that Libevent
doesn't actually need for anything) it really doesn't belong exposed
from Libevent.  [We don't want to mission-creep till we're a glib or
an apr or anspr.]  Since it was only introduced in 2.0.1-alpha, we
could remove it without breaking backward compatibility with any

SSL support

Hi, there,

I'm really interested in getting SSL support for evhttp and I saw there was
a patch doing that. Anyone has a pointer?

Thanks!

-Haiping

***********************************************************************
To unsubscribe, send an e-mail to majordomo <at> freehaven.net with
unsubscribe libevent-users    in the body.

Closing bufferevent

ANN: libevent-2.0.4-alpha released

Hi everyone,

please, help us test a new alpha release of libevent 2.0.   You can
download the release from

 http://www.monkey.org/~provos/libevent-2.0.4-alpha.tar.gz
 http://www.monkey.org/~provos/libevent-2.0.4-alpha.tar.gz.sig

Do not forget to validate the signature.   As it has been over 3
months since the last release, there have been a large number of
changes.   The complete change list is available at a really long URL:

 http://bit.ly/bnH6OT

Some of the feature improvements include:

- bufferevents can now be rate limited
- http connections can now resolve host names asynchronously
- a facility for lock debugging
- arc4random() for evdns

However, we (that means mostly Nick) have also made a large number of
bug fixes and stability improvements across many platforms.    Many
thanks to everyone who helped by providing bug reports and patches
including Brodie Thiesfield, Dagobert Michelsen, Evan Jones, Joachim
Bauch, Pavel Plesov, Roman Puls, Sebastian Hahn, William Ahern,
Yasuoka Masahiko and Zhuang Yuyao.

Thanks,
 Niels.
***********************************************************************
To unsubscribe, send an e-mail to majordomo <at> freehaven.net with
unsubscribe libevent-users    in the body.

Fun facts about Libevent 2.0.4-alpha

Hi, all!

As a followup to Niels's announcement, I'd like to go into a little
more detail about what's up with the 2.0.4-alpha release.

STATISTICS AND TRIVIA:
* Libevent 2.0.4-alpha is about 4000 lines longer than Libevent
2.0.3-alpha.  about 1700 of these are in tests and samples; the rest
is in actual code.

* Our unit test coverage has dipped a bit; in 2.0.3-alpha we were
around 80.5%; now we're around 79.3% coverage.  (Both numbers are from
my Linux box.)

* About 58% of Libevent's public functions, structures, and defines
are now documented in my reference manual at
http://www.wangafu.net/~nickm/libevent-book/ .

WHAT'S NEW:

For a full list of changes, see the ChangeLog on the website and
distributed with Libevent 2.0.4-alpha.  I'm only mentioning the stuff
I find interesting here.

* There are a ton of bugfixes, including a few memory corruption
issues and race conditions.  Everybody who's been writing
multithreaded code should upgrade.
* Bufferevents now support rate-limiting.  You can rate-limit
bufferevents individually, or in groups.  Right now, this feature is
underdocumented, since I suspect we'll want to tweak the APIs a bit
before 2.0.x goes stable.
* Our evdns_getaddrinfo()  and evutil_getaddrinfo() functions now
support looking at the /etc/hosts file, or local equivalent.
* The http subsystem can now use evdns to do non-blocking hostname
lookups for outgoing connections.
* Libevent backends can now use a "changelist" feature to minimize
calls to functions like epoll() and argument lengths to functions like
kqueue().  Right now the epoll and kqueue backends use this feature;
somebody could port evport and devpoll to do so as well.
* There's a new event_get_struct_event_size() function so you can use
event_assign() and friends without losing binary compatibility.
* There are new event_get_callback(), event_get_callback_arg()
functions to expose the remaining assigned members of an event, and an
event_get_assignment() function to get every member of an event at
once.
* There's now support for running Libevent in a "debug mode" that can
use a bit more memory and time, but detect some common programming
errors.  See the documentation for event_enable_debug_mode() for more
information.
* Libevent now uses a secure PRNG for the entropy that evdns needs to
be secure.  This is arc4random() on platforms that provide it, and our
own copy of arc4random() on platforms that don't.  You no longer need
to replace the evdns transaction ID or random_bytes functions for
security.

WHAT'S GONE OR CHANGED:

* If you have been manually providing locking functions with
evthread_set_lock_create_callback and evthread_set_locking_callback,
those functions are now deprecated.  You should use the more
future-proof evthread_set_lock_callbacks() function instead.  (Or you
could just use evthread_use_windows_threads() or
evthread_use_pthreads(), and not worry about low-level locking details
at all.)
* The EVENT_FD and EVENT_SIGNAL macros are now deprecated.
* Our vcproj files were completely incorrect; they built the wrong
files with the wrong options.  Nobody on the development team uses the
MSVC IDE, so   instead we're shipping makefiles for use with nmake.
These we can actually inspect by hand, and use reproducibly.
* We try to keep backward source compatibility with pre-2.0 versions
of Libevent.  Once Libevent 2.0.x is stable, we'll keep backward
compatibility with that too.  While 2.0.x is in alpha, though, we
occasionally change or remove APIs introduced in earlier 2.0.x alphas
so that we don't have to keep supporting bad ideas indefinitely.
   * The evhttp_connection_base_new() function now takes an optional
evdns_base argument.
   * The flags argument of evdns_base_set_option() has been removed.
   * The EVUTIL_CHECK_FMT macro has been removed.

hth,
--

-- 
Nick
***********************************************************************
To unsubscribe, send an e-mail to majordomo <at> freehaven.net with
unsubscribe libevent-users    in the body.

Closing bufferevents

Re: SSL support

On Tue, Mar 2, 2010 at 12:56 AM, Haiping Zhao <hzhao <at> facebook.com> wrote:
> Hi, there,
>
> I'm really interested in getting SSL support for evhttp and I saw there was
> a patch doing that. Anyone has a pointer?

Hi there!

The "ark" project patched the old Libevent 1.x http code to provide
SSL support  You can see some discussion at [1] and [2].  The URL in
those posts (http://www.tecsiel.it/ark) seems to be down right now; I
don't know whether that's temporary or permanent.

Libevent 2.0.x has integrated SSL support in its bufferevent layer
(currently using OpenSSL, but it could be extended to use other
libraries in the future).    The evhttp.c code doesn't use it right
now, but it would be neat to have a good patch for it.  The first step
would probably be figuring out the correct interface.  Niels, do you
have any suggestions here?

yrs,
--

-- 
Nick
***********************************************************************
To unsubscribe, send an e-mail to majordomo <at> freehaven.net with
unsubscribe libevent-users    in the body.

Re: Fun facts about Libevent 2.0.4-alpha

On Wed, Mar 03, 2010 at 04:38:59PM -0500, Nick Mathewson wrote:
>
> * Libevent now uses a secure PRNG for the entropy that evdns needs to
> be secure.  This is arc4random() on platforms that provide it, and our
> own copy of arc4random() on platforms that don't.  You no longer need
> to replace the evdns transaction ID or random_bytes functions for
> security.
> 

I noticed that the compat version will expect to open a /dev randomness
device to stir. But this looks like it will fail for daemons which chroot. A
quick grep looks like arc4random_stir() isn't called anywhere (i.e. from a
libevent init routine).

Two possible solution:

(1) call arc4random_stir() from something which it is reasonable to expect
the application to call before chroot'ing.

and/or

(2) as Linux is probably by far the most common environment for libevent,
first try to grab bytes from sysctl. here's the relevant snippet from my own
portable arc4random implemention:

  int mib[] = { CTL_KERN, KERN_RANDOM, RANDOM_UUID };
  unsigned char uuid[128];
  size_t len, n;

  for (len = 0; len < sizeof uuid; len += n) {
    n = sizeof uuid - len;

    if (0 != sysctl(mib, sizeof mib / sizeof mib[0], &uuid[len], &n, (void*)0, 0))
      break;
  }

  for (n = 0; n < len && n < sizeof rnd; n++)
    rnd.bytes[n] ^= uuid[n];

Interestingly, arc4random on FreeBSD just uses /dev/urandom, so at the very
least #1 should be used. But, that doesn't fix the re-seeding problem, and
at least on Linux that can be addressed like above.

***********************************************************************
To unsubscribe, send an e-mail to majordomo <at> freehaven.net with
unsubscribe libevent-users    in the body.

Re: Fun facts about Libevent 2.0.4-alpha

Hi,

Thanks for your work!

Another interesting fact about this release is that libevent-2 now
passes almost all of gevent's (www.gevent.org) test suite. The
previous alpha had a few failures in http server tests but 2.0.4 fixed
them!

There is one test that is still failing, but it's relatively minor:

In 1.4, if you send a request followed by some junk data, like this

       request = '\r\n'.join((
            'POST / HTTP/1.0',
            'Host: localhost',
            'Content-Length: 3',
            '',
            'a=a'))
        fd.write(request)

        # send some junk after the actual request
        fd.write('01234567890123456789')

then libevent will call the user's callback, ignoring the junk.

In 2.0.4 this request does not trigger a callback.

On Thu, Mar 4, 2010 at 3:38 AM, Nick Mathewson <nickm <at> freehaven.net> wrote:
> Hi, all!
>
> As a followup to Niels's announcement, I'd like to go into a little
> more detail about what's up with the 2.0.4-alpha release.
>
> STATISTICS AND TRIVIA:
> * Libevent 2.0.4-alpha is about 4000 lines longer than Libevent
> 2.0.3-alpha.  about 1700 of these are in tests and samples; the rest
> is in actual code.
>
> * Our unit test coverage has dipped a bit; in 2.0.3-alpha we were
> around 80.5%; now we're around 79.3% coverage.  (Both numbers are from
> my Linux box.)
>
> * About 58% of Libevent's public functions, structures, and defines
> are now documented in my reference manual at
> http://www.wangafu.net/~nickm/libevent-book/ .
>
> WHAT'S NEW:
>
> For a full list of changes, see the ChangeLog on the website and
> distributed with Libevent 2.0.4-alpha.  I'm only mentioning the stuff
> I find interesting here.
>
> * There are a ton of bugfixes, including a few memory corruption
> issues and race conditions.  Everybody who's been writing
> multithreaded code should upgrade.
> * Bufferevents now support rate-limiting.  You can rate-limit
> bufferevents individually, or in groups.  Right now, this feature is
> underdocumented, since I suspect we'll want to tweak the APIs a bit
> before 2.0.x goes stable.
> * Our evdns_getaddrinfo()  and evutil_getaddrinfo() functions now
> support looking at the /etc/hosts file, or local equivalent.
> * The http subsystem can now use evdns to do non-blocking hostname
> lookups for outgoing connections.
> * Libevent backends can now use a "changelist" feature to minimize
> calls to functions like epoll() and argument lengths to functions like
> kqueue().  Right now the epoll and kqueue backends use this feature;
> somebody could port evport and devpoll to do so as well.
> * There's a new event_get_struct_event_size() function so you can use
> event_assign() and friends without losing binary compatibility.
> * There are new event_get_callback(), event_get_callback_arg()
> functions to expose the remaining assigned members of an event, and an
> event_get_assignment() function to get every member of an event at
> once.
> * There's now support for running Libevent in a "debug mode" that can
> use a bit more memory and time, but detect some common programming
> errors.  See the documentation for event_enable_debug_mode() for more
> information.
> * Libevent now uses a secure PRNG for the entropy that evdns needs to
> be secure.  This is arc4random() on platforms that provide it, and our
> own copy of arc4random() on platforms that don't.  You no longer need
> to replace the evdns transaction ID or random_bytes functions for
> security.
>
> WHAT'S GONE OR CHANGED:
>
> * If you have been manually providing locking functions with
> evthread_set_lock_create_callback and evthread_set_locking_callback,
> those functions are now deprecated.  You should use the more
> future-proof evthread_set_lock_callbacks() function instead.  (Or you
> could just use evthread_use_windows_threads() or
> evthread_use_pthreads(), and not worry about low-level locking details
> at all.)
> * The EVENT_FD and EVENT_SIGNAL macros are now deprecated.
> * Our vcproj files were completely incorrect; they built the wrong
> files with the wrong options.  Nobody on the development team uses the
> MSVC IDE, so   instead we're shipping makefiles for use with nmake.
> These we can actually inspect by hand, and use reproducibly.
> * We try to keep backward source compatibility with pre-2.0 versions
> of Libevent.  Once Libevent 2.0.x is stable, we'll keep backward
> compatibility with that too.  While 2.0.x is in alpha, though, we
> occasionally change or remove APIs introduced in earlier 2.0.x alphas
> so that we don't have to keep supporting bad ideas indefinitely.
>   * The evhttp_connection_base_new() function now takes an optional
> evdns_base argument.
>   * The flags argument of evdns_base_set_option() has been removed.
>   * The EVUTIL_CHECK_FMT macro has been removed.
>
>
> hth,
> --
> Nick
> ***********************************************************************
> To unsubscribe, send an e-mail to majordomo <at> freehaven.net with
> unsubscribe libevent-users    in the body.
>
***********************************************************************
To unsubscribe, send an e-mail to majordomo <at> freehaven.net with
unsubscribe libevent-users    in the body.

arc4random and chroot [was Re: Fun facts about Libevent 2.0.4-alpha]

On Wed, Mar 3, 2010 at 6:08 PM, William Ahern
<william <at> 25thandclement.com> wrote:
> On Wed, Mar 03, 2010 at 04:38:59PM -0500, Nick Mathewson wrote:
>>
>> * Libevent now uses a secure PRNG for the entropy that evdns needs to
>> be secure.  This is arc4random() on platforms that provide it, and our
>> own copy of arc4random() on platforms that don't.  You no longer need
>> to replace the evdns transaction ID or random_bytes functions for
>> security.
>>
>
> I noticed that the compat version will expect to open a /dev randomness
> device to stir. But this looks like it will fail for daemons which chroot. A
> quick grep looks like arc4random_stir() isn't called anywhere (i.e. from a
> libevent init routine).
>
> Two possible solution:
>
> (1) call arc4random_stir() from something which it is reasonable to expect
> the application to call before chroot'ing.
>
> and/or
>
> (2) as Linux is probably by far the most common environment for libevent,
> first try to grab bytes from sysctl. here's the relevant snippet from my own
> portable arc4random implemention:

I'm not sure that the first approach is feasible; if you're using
libevent and chrooting, there really is no Libevent function that
you're guaranteed to call before a chroot.  I've decided to go with a
mixed approach based on your 2 and an additional 3.

(2') Try the Linux sysctl as well as the OpenBSD sysctl as well as
/dev/urandom on Unix boxen (where chroot matters).

(3) Tell people that if they need to chroot and they need to use
libevent in a way that requires a strong RNG, they need to initialize
the RNG before they chroot.  They can do this by calling
evutil_secure_rng_init() and checking that the return value is zero.

I've implemented both of these in a git branch called "arc4seed" at my
personal repository at git://github.com/nmathewson/Libevent.git .
Please have a look and let me know what you think?  I've only tried it
out on my Linux desktop; the BSD code (which uses the KERN_ARAND
syscall) might need some love, and I might even have broken the
windows stuff somehow.

yrs,
--

-- 
Nick
***********************************************************************
To unsubscribe, send an e-mail to majordomo <at> freehaven.net with
unsubscribe libevent-users    in the body.