Sarkis Varozian | 3 Mar 23:24 2015
Picon

Request Queueing after deploy + USR2 restart

We have a rails application with the following unicorn.rb:
http://goo.gl/qZ5NLn

When we deploy to the application, a USR2 signal is sent to the unicorn
master which spins up a new master and we use the before_fork in the
unicorn.rb config above to send signals to the old master as the new
workers come online.

I've been trying to debug a weird issue that manifests as "Request
Queueing" in our Newrelic APM. The graph shows what happens after a
deployment (represented by the vertical lines). Here is the graph:
http://goo.gl/iFZPMv . As you see from the graph, it is inconsistent -
there is always a latency spike - however, at times Request Queueing is
higher than previous deploys.

Any ideas on what exactly is going on here? Any suggestions on
tools/profilers to use to get to the bottom of this? Should we expect this
to happen on each deploy?

Thanks,

--

-- 
*Sarkis Varozian*
svarozian <at> gmail.com

Eric Wong | 2 Mar 21:34 2015

[RFC] http: remove experimental dechunk! method

It was never used anywhere AFAIK and wastes precious bytes.
---
 RFC since this didn't go through a deprecation period, but it's
 never been documented or heavily publicized (but that goes for
 this entire project :P)

 ext/unicorn_http/unicorn_http.rl | 29 ---------------------------
 test/unit/test_http_parser_ng.rb | 43 ----------------------------------------
 2 files changed, 72 deletions(-)

diff --git a/ext/unicorn_http/unicorn_http.rl b/ext/unicorn_http/unicorn_http.rl
index 03b4726..4254540 100644
--- a/ext/unicorn_http/unicorn_http.rl
+++ b/ext/unicorn_http/unicorn_http.rl
 <at>  <at>  -590,34 +590,6  <at>  <at>  static VALUE HttpParser_clear(VALUE self)
   return self;
 }

-/**
- * call-seq:
- *    parser.dechunk! => parser
- *
- * Resets the parser to a state suitable for dechunking response bodies
- *
- */
-static VALUE HttpParser_dechunk_bang(VALUE self)
-{
-  struct http_parser *hp = data_get(self);
-
-  http_parser_init(hp);
(Continue reading)

Eric Wong | 2 Mar 21:32 2015

[PATCH] http: remove deprecated reset method

We use the `clear' method everywhere nowadays.
---
 ext/unicorn_http/unicorn_http.rl | 22 ----------------------
 1 file changed, 22 deletions(-)

diff --git a/ext/unicorn_http/unicorn_http.rl b/ext/unicorn_http/unicorn_http.rl
index 932d259..03b4726 100644
--- a/ext/unicorn_http/unicorn_http.rl
+++ b/ext/unicorn_http/unicorn_http.rl
 <at>  <at>  -618,27 +618,6  <at>  <at>  static VALUE HttpParser_dechunk_bang(VALUE self)
   return self;
 }

-/**
- * call-seq:
- *    parser.reset => nil
- *
- * Resets the parser to it's initial state so that you can reuse it
- * rather than making new ones.
- *
- * This method is deprecated and to be removed in Unicorn 4.x
- */
-static VALUE HttpParser_reset(VALUE self)
-{
-  static int warned;
-
-  if (!warned) {
-    rb_warn("Unicorn::HttpParser#reset is deprecated; "
-            "use Unicorn::HttpParser#clear instead");
-  }
(Continue reading)

Eric Wong | 2 Mar 21:24 2015

[PATCH] save about 200 bytes of memory on x86-64

Empty classes do not need a heavy class definition scope.
---
 lib/unicorn.rb | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/lib/unicorn.rb b/lib/unicorn.rb
index dd02761..467245d 100644
--- a/lib/unicorn.rb
+++ b/lib/unicorn.rb
 <at>  <at>  -21,8 +21,7  <at>  <at>  module Unicorn
   # since there is nothing in the application stack that is responsible
   # for client shutdowns/disconnects.  This exception is visible to Rack
   # applications unless PrereadInput middleware is loaded.
-  class ClientShutdown < EOFError
-  end
+  ClientShutdown = Class.new(EOFError)

   # :stopdoc:

--

-- 
EW

Eric Wong | 18 Feb 10:36 2015

[PATCH] explain 11 byte magic number for self-pipe

Oops, this should've been explained long ago but apparently not.

In response to a comment on
http://www.sitepoint.com/the-self-pipe-trick-explained/

> Does anybody know why both unicorn and foreman read 11 bytes from
> self-pipe?

Unfortunately I couldn't find a way to comment on the site on a
JavaScript-free browser nor does it seem possible without
registering.

Again, anybody can send plain-text mail to:
unicorn-public <at> bogomips.org

No registration, no real name policy, no terms-of-service, just
plain-text.  Feel free to use Tor, mixmaster or any anonymity
service, too.
---
 lib/unicorn/http_server.rb | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/lib/unicorn/http_server.rb b/lib/unicorn/http_server.rb
index 015bd94..683eb82 100644
--- a/lib/unicorn/http_server.rb
+++ b/lib/unicorn/http_server.rb
 <at>  <at>  -370,6 +370,10  <at>  <at>  class Unicorn::HttpServer
   # wait for a signal hander to wake us up and then consume the pipe
   def master_sleep(sec)
     IO.select([  <at> self_pipe[0] ], nil, nil, sec) or return
(Continue reading)

Steven Stewart-Gallus | 16 Feb 03:19 2015
Picon

Bug, Unicorn can drop signals in extreme conditions

Hello,

While reading the article at
http://www.sitepoint.com/the-self-pipe-trick-explained/ I realized
that the signal handling code shown there and of the same type in your
application is broken.  As you have a single nonblocking pipe in your
application you can drop signals entirely (and not just fold multiple
signals of the same type into one).  The simplest fix is to use a pipe
for each individual type of signal or to just use pselect or similar.
I'm pretty sure that there is a way to use a single pipe but it seems
complicated and very difficult to implement correctly.

Thank you,
Steven Stewart-Gallus

Eric Wong | 12 Feb 20:09 2015

[PATCH] http_server: favor ivars over constants

In 1.9+ at least, instance variables use less space than constants
in class tables and bytecode, leading to ~700 byte reduction in
bytecode overhead on 64-bit and a reduction in constant table/entries
of the Unicorn::HttpServer class.
---
 lib/unicorn/http_server.rb | 77 +++++++++++++++++++++-------------------------
 1 file changed, 35 insertions(+), 42 deletions(-)

diff --git a/lib/unicorn/http_server.rb b/lib/unicorn/http_server.rb
index f0216d0..278b469 100644
--- a/lib/unicorn/http_server.rb
+++ b/lib/unicorn/http_server.rb
 <at>  <at>  -21,32 +21,12  <at>  <at>  class Unicorn::HttpServer
   include Unicorn::SocketHelper
   include Unicorn::HttpResponse

-  # backwards compatibility with 1.x
-  Worker = Unicorn::Worker
-
   # all bound listener sockets
   LISTENERS = []

   # listeners we have yet to bind
   NEW_LISTENERS = []

-  # This hash maps PIDs to Workers
-  WORKERS = {}
-
-  # We use SELF_PIPE differently in the master and worker processes:
-  #
(Continue reading)

Eric Wong | 10 Feb 18:06 2015

[PATCH] ISSUES: add section for bugs in other projects

This is not anything new, just documenting what has been going
on since the beginning.

There's been a small number of generic networking (or mm) bugs in
the kernel which affect unicorn, but are usually found and fixed
with more popular, non-Ruby servers, first.

Aside from generic performance problems, I don't think there's ever
been a glibc bug which affected unicorn.
---
 ISSUES | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)

diff --git a/ISSUES b/ISSUES
index f66d14a..343cab4 100644
--- a/ISSUES
+++ b/ISSUES
 <at>  <at>  -18,6 +18,36  <at>  <at>  instead and your issue will be handled discreetly.
 If you don't get a response within a few days, we may have forgotten
 about it so feel free to ask again.

+== Bugs in related projects
+
+unicorn is sometimes affected by bugs in its dependencies.  Bugs
+triggered by unicorn in mainline Ruby, rack, GNU C library (glibc),
+or the Linux kernel will be reported upstream and fixed.
+
+For bugs in Ruby itself, we may forward bugs to
+https://bugs.ruby-lang.org/ and discuss+fix them on the ruby-core
+list at mailto:ruby-core <at> ruby-lang.org
(Continue reading)

Eric Wong | 9 Feb 10:12 2015

[PATCH 1/2] const: drop constants used by Rainbows!

Rainbows! (in maintenance mode) will need to define it's own
constants in the future.  We'll trim down our constant usage in
subsequent commits as we take advantage of Ruby VM improvements.
---
 lib/unicorn/const.rb | 8 --------
 1 file changed, 8 deletions(-)

diff --git a/lib/unicorn/const.rb b/lib/unicorn/const.rb
index 26fa62b..e24b511 100644
--- a/lib/unicorn/const.rb
+++ b/lib/unicorn/const.rb
 <at>  <at>  -25,14 +25,6  <at>  <at>  module Unicorn::Const
   MAX_BODY = 1024 * 112

   # :stopdoc:
-  # common errors we'll send back
-  # (N.B. these are not used by unicorn, but we won't drop them until
-  #  unicorn 5.x to avoid breaking Rainbows!).
-  ERROR_400_RESPONSE = "HTTP/1.1 400 Bad Request\r\n\r\n"
-  ERROR_414_RESPONSE = "HTTP/1.1 414 Request-URI Too Long\r\n\r\n"
-  ERROR_413_RESPONSE = "HTTP/1.1 413 Request Entity Too Large\r\n\r\n"
-  ERROR_500_RESPONSE = "HTTP/1.1 500 Internal Server Error\r\n\r\n"
-
   EXPECT_100_RESPONSE = "HTTP/1.1 100 Continue\r\n\r\n"
   EXPECT_100_RESPONSE_SUFFIXED = "100 Continue\r\n\r\nHTTP/1.1 "

--

-- 
EW

(Continue reading)

Eric Wong | 6 Feb 23:50 2015

[PATCH] favor "a.b(&:c)" form over "a.b { |x| x.c }"

The former is shorter Ruby code and also generates smaller
bytecode.
---
 lib/unicorn/http_server.rb | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/unicorn/http_server.rb b/lib/unicorn/http_server.rb
index 895f56d..c44a71e 100644
--- a/lib/unicorn/http_server.rb
+++ b/lib/unicorn/http_server.rb
 <at>  <at>  -69,7 +69,7  <at>  <at>  class Unicorn::HttpServer
   #
   #   Unicorn::HttpServer::START_CTX[0] = "/home/bofh/2.2.0/bin/unicorn"
   START_CTX = {
-    :argv => ARGV.map { |arg| arg.dup },
+    :argv => ARGV.map(&:dup),
     0 => $0.dup,
   }
   # We favor ENV['PWD'] since it is (usually) symlink aware for Capistrano
 <at>  <at>  -483,7 +483,7  <at>  <at>  class Unicorn::HttpServer
   end

   def after_fork_internal
-    SELF_PIPE.each { |io| io.close }.clear # this is master-only, now
+    SELF_PIPE.each(&:close).clear # this is master-only, now
      <at> ready_pipe.close if  <at> ready_pipe
     Unicorn::Configurator::RACKUP.clear
      <at> ready_pipe =  <at> init_listeners =  <at> before_exec =  <at> before_fork = nil
--

-- 
EW
(Continue reading)

Eric Wong | 6 Feb 23:34 2015

[PATCH] test_socket_helper: do not depend on SO_REUSEPORT

Older Rubies (2.0) may not define SO_REUSEPORT even if the
kernel and libc support it
---
 test/unit/test_socket_helper.rb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/test/unit/test_socket_helper.rb b/test/unit/test_socket_helper.rb
index 8b09198..7526e82 100644
--- a/test/unit/test_socket_helper.rb
+++ b/test/unit/test_socket_helper.rb
 <at>  <at>  -193,5 +193,5  <at>  <at>  class TestSocketHelper < Test::Unit::TestCase
     assert_operator cur, :>, 0
   rescue Errno::ENOPROTOOPT
     # kernel does not support SO_REUSEPORT (older Linux)
-  end
+  end if defined?(Socket::SO_REUSEPORT)
 end
--

-- 
EW


Gmane