headers
Yaroslav | 2 Dec 2011 00:55
Picon

NXWEB - ultra-fast web server based on libev

Hi,


I'd like to thank the authors of libev for very good product. Based on it I have developed NXWEB web server, which I'd like to present to community. Here are the highlights:

What NXWEB offers:

  • good performance; see benchmarks at project page
  • can serve thousands concurrent requests
  • small memory footprint
  • multi-threaded model designed to scale
  • exceptionally light code base
  • simple API
  • decent HTTP protocol handling
  • keep-alive connections
  • non-blocking sendfile support
  • modular design for developers
  • can be run as daemon; relaunches itself on error
  • open source

Limitations:

  • no SSL support (yet)
  • only tested on Linux

Architecture

Main thread binds TCP port, accepts connections and distributes them among network threads.

Network threads work in non-blocking fashion (using libev) handling HTTP protocol exchange. There should be no need in using more network threads than the number of CPU cores you have, as every thread is very efficient and can easily handle thousands of concurrent connections.

After receiving complete HTTP request network thread finds and invokes application's URI handler. The invocation could happen within network thread (should only be used for quick non-blocking handlers) or within worker thread (should be used for slower or blocking handlers).

Worker threads are organized in pool. Each thread takes new job from queue, invokes URI handler, signals network thread after handler completes the job. If your handler is slow, you need to configure more worker threads.

Numbers of network and worker threads are configured via #define directives.

-----

NXWEB project page: https://bitbucket.org/yarosla/nxweb/

_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Permalink | Reply | 9 comments |
headers
Rodrigo Campos | 2 Dec 2011 03:04
Picon

Re: NXWEB - ultra-fast web server based on libev

On Fri, Dec 02, 2011 at 03:55:27AM +0400, Yaroslav wrote:
> Hi,
> 
> I'd like to thank the authors of libev for very good product. Based on it I
> have developed NXWEB web server, which I'd like to present to community. Here
> are the highlights:
> 
> NXWEB project page: https://bitbucket.org/yarosla/nxweb/

Sounds interesting, but the project page asks for a login. It seems I can not
see anything without an account :(

Thanks,
Rodrigo

_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Permalink | Reply | 8 comments |
headers
Chris Brody | 2 Dec 2011 09:13
Picon

Re: NXWEB - ultra-fast web server based on libev

Yeah I created an account with Bitbucket and still could not get access to the Wiki.


Can you please make this project public if you want to announce it?

On Fri, Dec 2, 2011 at 3:04 AM, Rodrigo Campos <rodrigo <at> sdfg.com.ar> wrote:
On Fri, Dec 02, 2011 at 03:55:27AM +0400, Yaroslav wrote:
> Hi,
>
> I'd like to thank the authors of libev for very good product. Based on it I
> have developed NXWEB web server, which I'd like to present to community. Here
> are the highlights:
>
> NXWEB project page: https://bitbucket.org/yarosla/nxweb/

Sounds interesting, but the project page asks for a login. It seems I can not
see anything without an account :(





Thanks,
Rodrigo

_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev 

_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Permalink | Reply | 7 comments |
headers
Yaroslav | 2 Dec 2011 09:58
Picon

Re: NXWEB - ultra-fast web server based on libev

What a shame. Sorry about that. It is my first public repo at Bitbucket, I turned 'private' flag off, but did not check that I have to make wiki and issue tracker public as well.


It should be all OK now.

Sorry for the inconvenience.

On Fri, Dec 2, 2011 at 12:13 PM, Chris Brody <chris.brody <at> gmail.com> wrote:
Yeah I created an account with Bitbucket and still could not get access to the Wiki.

Can you please make this project public if you want to announce it?


On Fri, Dec 2, 2011 at 3:04 AM, Rodrigo Campos <rodrigo <at> sdfg.com.ar> wrote:
On Fri, Dec 02, 2011 at 03:55:27AM +0400, Yaroslav wrote:
> Hi,
>
> I'd like to thank the authors of libev for very good product. Based on it I
> have developed NXWEB web server, which I'd like to present to community. Here
> are the highlights:
>
> NXWEB project page: https://bitbucket.org/yarosla/nxweb/

Sounds interesting, but the project page asks for a login. It seems I can not
see anything without an account :(





Thanks,
Rodrigo

_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev


_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev

_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Permalink | Reply | 6 comments |
headers
Zabrane Mickael | 2 Dec 2011 11:56
Picon

Re: NXWEB - ultra-fast web server based on libev

Hi Yaroslav,

My first try to compile NxWeb (OSX 10.6.8 + libev4):

nxweb $  make
mkdir -p bin/Release
mkdir -p obj/Release/nxweb
gcc -c -o obj/Release/nxweb/main.o nxweb/main.c -pthread -Wno-strict-aliasing -O2 -s
nxweb/main.c:29:18: error: argp.h: No such file or directory
In file included from nxweb/nxweb_internal.h:30,
                 from nxweb/main.c:35:
nxweb/nxweb.h:30:21: error: obstack.h: No such file or directory
In file included from nxweb/nxweb_internal.h:30,
                 from nxweb/main.c:35:
nxweb/nxweb.h:121: error: field ‘data’ has incomplete type
nxweb/nxweb.h:122: error: field ‘user_data’ has incomplete type
nxweb/main.c:123: error: array type has incomplete element type
nxweb/main.c:140: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘parse_opt’
nxweb/main.c:165: error: variable ‘argp’ has initializer but incomplete type
nxweb/main.c:165: warning: excess elements in struct initializer
nxweb/main.c:165: warning: (near initialization for ‘argp’)
nxweb/main.c:165: error: ‘parse_opt’ undeclared here (not in a function)
nxweb/main.c:165: warning: excess elements in struct initializer
nxweb/main.c:165: warning: (near initialization for ‘argp’)
nxweb/main.c:165: warning: excess elements in struct initializer
nxweb/main.c:165: warning: (near initialization for ‘argp’)
nxweb/main.c:165: warning: excess elements in struct initializer
nxweb/main.c:165: warning: (near initialization for ‘argp’)
make: *** [obj/Release/nxweb/main.o] Error 1


I think you should update your benchmark to use weighttp:
As pointed by G-WAN's author, ab performs badly on multi-cores.

Regards,
Zabrane

On Dec 2, 2011, at 9:58 AM, Yaroslav wrote:

What a shame. Sorry about that. It is my first public repo at Bitbucket, I turned 'private' flag off, but did not check that I have to make wiki and issue tracker public as well.

It should be all OK now.

Sorry for the inconvenience.

On Fri, Dec 2, 2011 at 12:13 PM, Chris Brody <chris.brody <at> gmail.com> wrote:
Yeah I created an account with Bitbucket and still could not get access to the Wiki.

Can you please make this project public if you want to announce it?


On Fri, Dec 2, 2011 at 3:04 AM, Rodrigo Campos <rodrigo <at> sdfg.com.ar> wrote:
On Fri, Dec 02, 2011 at 03:55:27AM +0400, Yaroslav wrote:
> Hi,
>
> I'd like to thank the authors of libev for very good product. Based on it I
> have developed NXWEB web server, which I'd like to present to community. Here
> are the highlights:
>
> NXWEB project page: https://bitbucket.org/yarosla/nxweb/

Sounds interesting, but the project page asks for a login. It seems I can not
see anything without an account :(





Thanks,
Rodrigo

_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev


_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev

_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev


_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Permalink | Reply | 5 comments |
headers
Yaroslav | 2 Dec 2011 12:14
Picon

Re: NXWEB - ultra-fast web server based on libev

Hi Mickael,


I mentioned in limitations that it has only been tested on Linux. I can't test it on OSX unfortunately, and frankly speaking portability was not my primary goal. At least not at this stage. Hosting providers mostly offer Linux, never heard about OSX.

But still I think it should be possible to port it to OSX, it's not Windows anyway.

What causing compile errors is the absense of argp and obstack in OSX's library. argp is not so critical for NXWEB - this is just command line parsing. While obstack is used extensively by the core.

Thanks for pointing to benchmarking tool. I will take a look at it. I wasn't satisfied by ab, but couldn't find anything better. httperf happened to be too complicated for my brain.

Yaroslav

On Fri, Dec 2, 2011 at 2:56 PM, Zabrane Mickael <zabrane3 <at> gmail.com> wrote:
Hi Yaroslav,

My first try to compile NxWeb (OSX 10.6.8 + libev4):

nxweb $  make
mkdir -p bin/Release
mkdir -p obj/Release/nxweb
gcc -c -o obj/Release/nxweb/main.o nxweb/main.c -pthread -Wno-strict-aliasing -O2 -s
nxweb/main.c:29:18: error: argp.h: No such file or directory
In file included from nxweb/nxweb_internal.h:30,
                 from nxweb/main.c:35:
nxweb/nxweb.h:30:21: error: obstack.h: No such file or directory
In file included from nxweb/nxweb_internal.h:30,
                 from nxweb/main.c:35:
nxweb/nxweb.h:121: error: field ‘data’ has incomplete type
nxweb/nxweb.h:122: error: field ‘user_data’ has incomplete type
nxweb/main.c:123: error: array type has incomplete element type
nxweb/main.c:140: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘parse_opt’
nxweb/main.c:165: error: variable ‘argp’ has initializer but incomplete type
nxweb/main.c:165: warning: excess elements in struct initializer
nxweb/main.c:165: warning: (near initialization for ‘argp’)
nxweb/main.c:165: error: ‘parse_opt’ undeclared here (not in a function)
nxweb/main.c:165: warning: excess elements in struct initializer
nxweb/main.c:165: warning: (near initialization for ‘argp’)
nxweb/main.c:165: warning: excess elements in struct initializer
nxweb/main.c:165: warning: (near initialization for ‘argp’)
nxweb/main.c:165: warning: excess elements in struct initializer
nxweb/main.c:165: warning: (near initialization for ‘argp’)
make: *** [obj/Release/nxweb/main.o] Error 1


I think you should update your benchmark to use weighttp:
As pointed by G-WAN's author, ab performs badly on multi-cores.

Regards,
Zabrane

On Dec 2, 2011, at 9:58 AM, Yaroslav wrote:

What a shame. Sorry about that. It is my first public repo at Bitbucket, I turned 'private' flag off, but did not check that I have to make wiki and issue tracker public as well.

It should be all OK now.

Sorry for the inconvenience.

On Fri, Dec 2, 2011 at 12:13 PM, Chris Brody <chris.brody <at> gmail.com> wrote:
Yeah I created an account with Bitbucket and still could not get access to the Wiki.

Can you please make this project public if you want to announce it?


On Fri, Dec 2, 2011 at 3:04 AM, Rodrigo Campos <rodrigo <at> sdfg.com.ar> wrote:
On Fri, Dec 02, 2011 at 03:55:27AM +0400, Yaroslav wrote:
> Hi,
>
> I'd like to thank the authors of libev for very good product. Based on it I
> have developed NXWEB web server, which I'd like to present to community. Here
> are the highlights:
>
> NXWEB project page: https://bitbucket.org/yarosla/nxweb/

Sounds interesting, but the project page asks for a login. It seems I can not
see anything without an account :(





Thanks,
Rodrigo

_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev


_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev

_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev 



_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Permalink | Reply | 4 comments |
headers
Yaroslav | 4 Dec 2011 01:25
Picon

Re: NXWEB - ultra-fast web server based on libev

Hi, everybody!

I tried to use weighttp for benchmarking but encountered some problems when using large number of concurrent connections in multi-threaded mode. For example:

   weighttp -c 10000 -n 100000 -t 4 -k http://...

This almost always produces quite a number of '104 Connection reset by peer' errors, many connections do not close, which means the test runs long, and measurements get poor. This is not specific to nxweb, same situation with all tested servers. I even tried it on different hardware with CentOS. Same situation. The interesting part is that when used in single thread mode (-t 1) there are no such errors even with large number of connections.

I've spent whole day digging the source of weighttp, even made some patches, but could not figure out the source of the problem.

Finally I've decided to write my own benchmarking tool, and successfully did it. With the same result though... As I also used libev.

Anyway I made some improvements over weighttp, namely:
- load is evenly distributed among threads, in weighttp one thread could finish much earlier than others
- I forcibly cut timed out connections, not allowing them to spoil the test

Otherwise it is the same efficient as weighttp. Need to mention that I have not implemented (yet) chunked encoding, custom headers and IPv6, which weighttp implements.

Here is the project page if anyone interested: https://bitbucket.org/yarosla/httpress/

I plan to redo all benchmarks for NXWEB in a few days. With the new tool results are much better now.

I wonder if anybody experienced same issues with hanging/reset connections with libev under high concurrency?

Regards,

Yaroslav

On Fri, Dec 2, 2011 at 3:14 PM, Yaroslav <yarosla <at> gmail.com> wrote:
Hi Mickael,

I mentioned in limitations that it has only been tested on Linux. I can't test it on OSX unfortunately, and frankly speaking portability was not my primary goal. At least not at this stage. Hosting providers mostly offer Linux, never heard about OSX.

But still I think it should be possible to port it to OSX, it's not Windows anyway.

What causing compile errors is the absense of argp and obstack in OSX's library. argp is not so critical for NXWEB - this is just command line parsing. While obstack is used extensively by the core.

Thanks for pointing to benchmarking tool. I will take a look at it. I wasn't satisfied by ab, but couldn't find anything better. httperf happened to be too complicated for my brain.

Yaroslav


On Fri, Dec 2, 2011 at 2:56 PM, Zabrane Mickael <zabrane3 <at> gmail.com> wrote:
Hi Yaroslav,

My first try to compile NxWeb (OSX 10.6.8 + libev4):

nxweb $  make
mkdir -p bin/Release
mkdir -p obj/Release/nxweb
gcc -c -o obj/Release/nxweb/main.o nxweb/main.c -pthread -Wno-strict-aliasing -O2 -s
nxweb/main.c:29:18: error: argp.h: No such file or directory
In file included from nxweb/nxweb_internal.h:30,
                 from nxweb/main.c:35:
nxweb/nxweb.h:30:21: error: obstack.h: No such file or directory
In file included from nxweb/nxweb_internal.h:30,
                 from nxweb/main.c:35:
nxweb/nxweb.h:121: error: field ‘data’ has incomplete type
nxweb/nxweb.h:122: error: field ‘user_data’ has incomplete type
nxweb/main.c:123: error: array type has incomplete element type
nxweb/main.c:140: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘parse_opt’
nxweb/main.c:165: error: variable ‘argp’ has initializer but incomplete type
nxweb/main.c:165: warning: excess elements in struct initializer
nxweb/main.c:165: warning: (near initialization for ‘argp’)
nxweb/main.c:165: error: ‘parse_opt’ undeclared here (not in a function)
nxweb/main.c:165: warning: excess elements in struct initializer
nxweb/main.c:165: warning: (near initialization for ‘argp’)
nxweb/main.c:165: warning: excess elements in struct initializer
nxweb/main.c:165: warning: (near initialization for ‘argp’)
nxweb/main.c:165: warning: excess elements in struct initializer
nxweb/main.c:165: warning: (near initialization for ‘argp’)
make: *** [obj/Release/nxweb/main.o] Error 1


I think you should update your benchmark to use weighttp:
As pointed by G-WAN's author, ab performs badly on multi-cores.

Regards,
Zabrane

On Dec 2, 2011, at 9:58 AM, Yaroslav wrote:

What a shame. Sorry about that. It is my first public repo at Bitbucket, I turned 'private' flag off, but did not check that I have to make wiki and issue tracker public as well.

It should be all OK now.

Sorry for the inconvenience.

On Fri, Dec 2, 2011 at 12:13 PM, Chris Brody <chris.brody <at> gmail.com> wrote:
Yeah I created an account with Bitbucket and still could not get access to the Wiki.

Can you please make this project public if you want to announce it?


On Fri, Dec 2, 2011 at 3:04 AM, Rodrigo Campos <rodrigo <at> sdfg.com.ar> wrote:
On Fri, Dec 02, 2011 at 03:55:27AM +0400, Yaroslav wrote:
> Hi,
>
> I'd like to thank the authors of libev for very good product. Based on it I
> have developed NXWEB web server, which I'd like to present to community. Here
> are the highlights:
>
> NXWEB project page: https://bitbucket.org/yarosla/nxweb/

Sounds interesting, but the project page asks for a login. It seems I can not
see anything without an account :(





Thanks,
Rodrigo

_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev


_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev

_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev 




_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Permalink | Reply | 3 comments |
headers
Zabrane Mickael | 4 Dec 2011 02:26
Picon

Re: NXWEB - ultra-fast web server based on libev

Hi Yaroslav,

I noticed te 104 error only when the server can't handle the large amount of connections correctly.

Did you check your ulimit settings?
You can refer to the excellent G-WAN doc explaining how to tweek your seetings at:

Let me give httpress a try, I'll be back with more info.

Keep up the good work man.

Regards,
Zabrane

On Dec 4, 2011, at 1:25 AM, Yaroslav wrote:

Hi, everybody!

I tried to use weighttp for benchmarking but encountered some problems when using large number of concurrent connections in multi-threaded mode. For example:

   weighttp -c 10000 -n 100000 -t 4 -k http://...

This almost always produces quite a number of '104 Connection reset by peer' errors, many connections do not close, which means the test runs long, and measurements get poor. This is not specific to nxweb, same situation with all tested servers. I even tried it on different hardware with CentOS. Same situation. The interesting part is that when used in single thread mode (-t 1) there are no such errors even with large number of connections.

I've spent whole day digging the source of weighttp, even made some patches, but could not figure out the source of the problem.

Finally I've decided to write my own benchmarking tool, and successfully did it. With the same result though... As I also used libev.

Anyway I made some improvements over weighttp, namely:
- load is evenly distributed among threads, in weighttp one thread could finish much earlier than others
- I forcibly cut timed out connections, not allowing them to spoil the test

Otherwise it is the same efficient as weighttp. Need to mention that I have not implemented (yet) chunked encoding, custom headers and IPv6, which weighttp implements.

Here is the project page if anyone interested: https://bitbucket.org/yarosla/httpress/

I plan to redo all benchmarks for NXWEB in a few days. With the new tool results are much better now.

I wonder if anybody experienced same issues with hanging/reset connections with libev under high concurrency?

Regards,

Yaroslav

On Fri, Dec 2, 2011 at 3:14 PM, Yaroslav <yarosla <at> gmail.com> wrote:
Hi Mickael,

I mentioned in limitations that it has only been tested on Linux. I can't test it on OSX unfortunately, and frankly speaking portability was not my primary goal. At least not at this stage. Hosting providers mostly offer Linux, never heard about OSX.

But still I think it should be possible to port it to OSX, it's not Windows anyway.

What causing compile errors is the absense of argp and obstack in OSX's library. argp is not so critical for NXWEB - this is just command line parsing. While obstack is used extensively by the core.

Thanks for pointing to benchmarking tool. I will take a look at it. I wasn't satisfied by ab, but couldn't find anything better. httperf happened to be too complicated for my brain.

Yaroslav


On Fri, Dec 2, 2011 at 2:56 PM, Zabrane Mickael <zabrane3 <at> gmail.com> wrote:
Hi Yaroslav,

My first try to compile NxWeb (OSX 10.6.8 + libev4):

nxweb $  make
mkdir -p bin/Release
mkdir -p obj/Release/nxweb
gcc -c -o obj/Release/nxweb/main.o nxweb/main.c -pthread -Wno-strict-aliasing -O2 -s
nxweb/main.c:29:18: error: argp.h: No such file or directory
In file included from nxweb/nxweb_internal.h:30,
                 from nxweb/main.c:35:
nxweb/nxweb.h:30:21: error: obstack.h: No such file or directory
In file included from nxweb/nxweb_internal.h:30,
                 from nxweb/main.c:35:
nxweb/nxweb.h:121: error: field ‘data’ has incomplete type
nxweb/nxweb.h:122: error: field ‘user_data’ has incomplete type
nxweb/main.c:123: error: array type has incomplete element type
nxweb/main.c:140: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘parse_opt’
nxweb/main.c:165: error: variable ‘argp’ has initializer but incomplete type
nxweb/main.c:165: warning: excess elements in struct initializer
nxweb/main.c:165: warning: (near initialization for ‘argp’)
nxweb/main.c:165: error: ‘parse_opt’ undeclared here (not in a function)
nxweb/main.c:165: warning: excess elements in struct initializer
nxweb/main.c:165: warning: (near initialization for ‘argp’)
nxweb/main.c:165: warning: excess elements in struct initializer
nxweb/main.c:165: warning: (near initialization for ‘argp’)
nxweb/main.c:165: warning: excess elements in struct initializer
nxweb/main.c:165: warning: (near initialization for ‘argp’)
make: *** [obj/Release/nxweb/main.o] Error 1


I think you should update your benchmark to use weighttp:
As pointed by G-WAN's author, ab performs badly on multi-cores.

Regards,
Zabrane

On Dec 2, 2011, at 9:58 AM, Yaroslav wrote:

What a shame. Sorry about that. It is my first public repo at Bitbucket, I turned 'private' flag off, but did not check that I have to make wiki and issue tracker public as well.

It should be all OK now.

Sorry for the inconvenience.

On Fri, Dec 2, 2011 at 12:13 PM, Chris Brody <chris.brody <at> gmail.com> wrote:
Yeah I created an account with Bitbucket and still could not get access to the Wiki.

Can you please make this project public if you want to announce it?


On Fri, Dec 2, 2011 at 3:04 AM, Rodrigo Campos <rodrigo <at> sdfg.com.ar> wrote:
On Fri, Dec 02, 2011 at 03:55:27AM +0400, Yaroslav wrote:
> Hi,
>
> I'd like to thank the authors of libev for very good product. Based on it I
> have developed NXWEB web server, which I'd like to present to community. Here
> are the highlights:
>
> NXWEB project page: https://bitbucket.org/yarosla/nxweb/

Sounds interesting, but the project page asks for a login. It seems I can not
see anything without an account :(





Thanks,
Rodrigo

_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev


_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev

_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev




_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev



_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Permalink | Reply | 2 comments |
headers
Yaroslav | 4 Dec 2011 15:00
Picon

Re: NXWEB - ultra-fast web server based on libev

My ulimit I think is high enough. It was 65536, now I checked with 131072. Same results. And with all three servers: nxweb, g-wan, nginx.


But after applying magic sysctl script error rate actually reduced. It looks frightening though.

On Sun, Dec 4, 2011 at 5:26 AM, Zabrane Mickael <zabrane3 <at> gmail.com> wrote:
Hi Yaroslav,

I noticed te 104 error only when the server can't handle the large amount of connections correctly.

Did you check your ulimit settings?
You can refer to the excellent G-WAN doc explaining how to tweek your seetings at:

Let me give httpress a try, I'll be back with more info.

Keep up the good work man.

Regards,
Zabrane

On Dec 4, 2011, at 1:25 AM, Yaroslav wrote:

Hi, everybody!

I tried to use weighttp for benchmarking but encountered some problems when using large number of concurrent connections in multi-threaded mode. For example:

   weighttp -c 10000 -n 100000 -t 4 -k http://...

This almost always produces quite a number of '104 Connection reset by peer' errors, many connections do not close, which means the test runs long, and measurements get poor. This is not specific to nxweb, same situation with all tested servers. I even tried it on different hardware with CentOS. Same situation. The interesting part is that when used in single thread mode (-t 1) there are no such errors even with large number of connections.

I've spent whole day digging the source of weighttp, even made some patches, but could not figure out the source of the problem.

Finally I've decided to write my own benchmarking tool, and successfully did it. With the same result though... As I also used libev.

Anyway I made some improvements over weighttp, namely:
- load is evenly distributed among threads, in weighttp one thread could finish much earlier than others
- I forcibly cut timed out connections, not allowing them to spoil the test

Otherwise it is the same efficient as weighttp. Need to mention that I have not implemented (yet) chunked encoding, custom headers and IPv6, which weighttp implements.

Here is the project page if anyone interested: https://bitbucket.org/yarosla/httpress/

I plan to redo all benchmarks for NXWEB in a few days. With the new tool results are much better now.

I wonder if anybody experienced same issues with hanging/reset connections with libev under high concurrency?

Regards,

Yaroslav

On Fri, Dec 2, 2011 at 3:14 PM, Yaroslav <yarosla <at> gmail.com> wrote:
Hi Mickael,

I mentioned in limitations that it has only been tested on Linux. I can't test it on OSX unfortunately, and frankly speaking portability was not my primary goal. At least not at this stage. Hosting providers mostly offer Linux, never heard about OSX.

But still I think it should be possible to port it to OSX, it's not Windows anyway.

What causing compile errors is the absense of argp and obstack in OSX's library. argp is not so critical for NXWEB - this is just command line parsing. While obstack is used extensively by the core.

Thanks for pointing to benchmarking tool. I will take a look at it. I wasn't satisfied by ab, but couldn't find anything better. httperf happened to be too complicated for my brain.

Yaroslav


On Fri, Dec 2, 2011 at 2:56 PM, Zabrane Mickael <zabrane3 <at> gmail.com> wrote:
Hi Yaroslav,

My first try to compile NxWeb (OSX 10.6.8 + libev4):

nxweb $  make
mkdir -p bin/Release
mkdir -p obj/Release/nxweb
gcc -c -o obj/Release/nxweb/main.o nxweb/main.c -pthread -Wno-strict-aliasing -O2 -s
nxweb/main.c:29:18: error: argp.h: No such file or directory
In file included from nxweb/nxweb_internal.h:30,
                 from nxweb/main.c:35:
nxweb/nxweb.h:30:21: error: obstack.h: No such file or directory
In file included from nxweb/nxweb_internal.h:30,
                 from nxweb/main.c:35:
nxweb/nxweb.h:121: error: field ‘data’ has incomplete type
nxweb/nxweb.h:122: error: field ‘user_data’ has incomplete type
nxweb/main.c:123: error: array type has incomplete element type
nxweb/main.c:140: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘parse_opt’
nxweb/main.c:165: error: variable ‘argp’ has initializer but incomplete type
nxweb/main.c:165: warning: excess elements in struct initializer
nxweb/main.c:165: warning: (near initialization for ‘argp’)
nxweb/main.c:165: error: ‘parse_opt’ undeclared here (not in a function)
nxweb/main.c:165: warning: excess elements in struct initializer
nxweb/main.c:165: warning: (near initialization for ‘argp’)
nxweb/main.c:165: warning: excess elements in struct initializer
nxweb/main.c:165: warning: (near initialization for ‘argp’)
nxweb/main.c:165: warning: excess elements in struct initializer
nxweb/main.c:165: warning: (near initialization for ‘argp’)
make: *** [obj/Release/nxweb/main.o] Error 1


I think you should update your benchmark to use weighttp:
As pointed by G-WAN's author, ab performs badly on multi-cores.

Regards,
Zabrane

On Dec 2, 2011, at 9:58 AM, Yaroslav wrote:

What a shame. Sorry about that. It is my first public repo at Bitbucket, I turned 'private' flag off, but did not check that I have to make wiki and issue tracker public as well.

It should be all OK now.

Sorry for the inconvenience.

On Fri, Dec 2, 2011 at 12:13 PM, Chris Brody <chris.brody <at> gmail.com> wrote:
Yeah I created an account with Bitbucket and still could not get access to the Wiki.

Can you please make this project public if you want to announce it?


On Fri, Dec 2, 2011 at 3:04 AM, Rodrigo Campos <rodrigo <at> sdfg.com.ar> wrote:
On Fri, Dec 02, 2011 at 03:55:27AM +0400, Yaroslav wrote:
> Hi,
>
> I'd like to thank the authors of libev for very good product. Based on it I
> have developed NXWEB web server, which I'd like to present to community. Here
> are the highlights:
>
> NXWEB project page: https://bitbucket.org/yarosla/nxweb/

Sounds interesting, but the project page asks for a login. It seems I can not
see anything without an account :(





Thanks,
Rodrigo

_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev


_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev

_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev




_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev 




_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Permalink | Reply | 1 comment |
headers
Yaroslav | 4 Dec 2011 23:08
Picon

Re: NXWEB - ultra-fast web server based on libev

I have published new benchmarks: https://bitbucket.org/yarosla/nxweb/wiki/Benchmarks

On Sun, Dec 4, 2011 at 6:00 PM, Yaroslav <yarosla <at> gmail.com> wrote:
My ulimit I think is high enough. It was 65536, now I checked with 131072. Same results. And with all three servers: nxweb, g-wan, nginx.

But after applying magic sysctl script error rate actually reduced. It looks frightening though.


On Sun, Dec 4, 2011 at 5:26 AM, Zabrane Mickael <zabrane3 <at> gmail.com> wrote:
Hi Yaroslav,

I noticed te 104 error only when the server can't handle the large amount of connections correctly.

Did you check your ulimit settings?
You can refer to the excellent G-WAN doc explaining how to tweek your seetings at:

Let me give httpress a try, I'll be back with more info.

Keep up the good work man.

Regards,
Zabrane

On Dec 4, 2011, at 1:25 AM, Yaroslav wrote:

Hi, everybody!

I tried to use weighttp for benchmarking but encountered some problems when using large number of concurrent connections in multi-threaded mode. For example:

   weighttp -c 10000 -n 100000 -t 4 -k http://...

This almost always produces quite a number of '104 Connection reset by peer' errors, many connections do not close, which means the test runs long, and measurements get poor. This is not specific to nxweb, same situation with all tested servers. I even tried it on different hardware with CentOS. Same situation. The interesting part is that when used in single thread mode (-t 1) there are no such errors even with large number of connections.

I've spent whole day digging the source of weighttp, even made some patches, but could not figure out the source of the problem.

Finally I've decided to write my own benchmarking tool, and successfully did it. With the same result though... As I also used libev.

Anyway I made some improvements over weighttp, namely:
- load is evenly distributed among threads, in weighttp one thread could finish much earlier than others
- I forcibly cut timed out connections, not allowing them to spoil the test

Otherwise it is the same efficient as weighttp. Need to mention that I have not implemented (yet) chunked encoding, custom headers and IPv6, which weighttp implements.

Here is the project page if anyone interested: https://bitbucket.org/yarosla/httpress/

I plan to redo all benchmarks for NXWEB in a few days. With the new tool results are much better now.

I wonder if anybody experienced same issues with hanging/reset connections with libev under high concurrency?

Regards,

Yaroslav

On Fri, Dec 2, 2011 at 3:14 PM, Yaroslav <yarosla <at> gmail.com> wrote:
Hi Mickael,

I mentioned in limitations that it has only been tested on Linux. I can't test it on OSX unfortunately, and frankly speaking portability was not my primary goal. At least not at this stage. Hosting providers mostly offer Linux, never heard about OSX.

But still I think it should be possible to port it to OSX, it's not Windows anyway.

What causing compile errors is the absense of argp and obstack in OSX's library. argp is not so critical for NXWEB - this is just command line parsing. While obstack is used extensively by the core.

Thanks for pointing to benchmarking tool. I will take a look at it. I wasn't satisfied by ab, but couldn't find anything better. httperf happened to be too complicated for my brain.

Yaroslav


On Fri, Dec 2, 2011 at 2:56 PM, Zabrane Mickael <zabrane3 <at> gmail.com> wrote:
Hi Yaroslav,

My first try to compile NxWeb (OSX 10.6.8 + libev4):

nxweb $  make
mkdir -p bin/Release
mkdir -p obj/Release/nxweb
gcc -c -o obj/Release/nxweb/main.o nxweb/main.c -pthread -Wno-strict-aliasing -O2 -s
nxweb/main.c:29:18: error: argp.h: No such file or directory
In file included from nxweb/nxweb_internal.h:30,
                 from nxweb/main.c:35:
nxweb/nxweb.h:30:21: error: obstack.h: No such file or directory
In file included from nxweb/nxweb_internal.h:30,
                 from nxweb/main.c:35:
nxweb/nxweb.h:121: error: field ‘data’ has incomplete type
nxweb/nxweb.h:122: error: field ‘user_data’ has incomplete type
nxweb/main.c:123: error: array type has incomplete element type
nxweb/main.c:140: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘parse_opt’
nxweb/main.c:165: error: variable ‘argp’ has initializer but incomplete type
nxweb/main.c:165: warning: excess elements in struct initializer
nxweb/main.c:165: warning: (near initialization for ‘argp’)
nxweb/main.c:165: error: ‘parse_opt’ undeclared here (not in a function)
nxweb/main.c:165: warning: excess elements in struct initializer
nxweb/main.c:165: warning: (near initialization for ‘argp’)
nxweb/main.c:165: warning: excess elements in struct initializer
nxweb/main.c:165: warning: (near initialization for ‘argp’)
nxweb/main.c:165: warning: excess elements in struct initializer
nxweb/main.c:165: warning: (near initialization for ‘argp’)
make: *** [obj/Release/nxweb/main.o] Error 1


I think you should update your benchmark to use weighttp:
As pointed by G-WAN's author, ab performs badly on multi-cores.

Regards,
Zabrane

On Dec 2, 2011, at 9:58 AM, Yaroslav wrote:

What a shame. Sorry about that. It is my first public repo at Bitbucket, I turned 'private' flag off, but did not check that I have to make wiki and issue tracker public as well.

It should be all OK now.

Sorry for the inconvenience.

On Fri, Dec 2, 2011 at 12:13 PM, Chris Brody <chris.brody <at> gmail.com> wrote:
Yeah I created an account with Bitbucket and still could not get access to the Wiki.

Can you please make this project public if you want to announce it?


On Fri, Dec 2, 2011 at 3:04 AM, Rodrigo Campos <rodrigo <at> sdfg.com.ar> wrote:
On Fri, Dec 02, 2011 at 03:55:27AM +0400, Yaroslav wrote:
> Hi,
>
> I'd like to thank the authors of libev for very good product. Based on it I
> have developed NXWEB web server, which I'd like to present to community. Here
> are the highlights:
>
> NXWEB project page: https://bitbucket.org/yarosla/nxweb/

Sounds interesting, but the project page asks for a login. It seems I can not
see anything without an account :(





Thanks,
Rodrigo

_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev


_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev

_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev




_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev 





_______________________________________________
libev mailing list
libev <at> lists.schmorp.de
http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
Permalink | Reply | 0 comments |

Gmane