Eric Wong | 24 Apr 05:02 2015

[PATCH 0/2] Rack::TempfileReaper support

These will be in the upcoming 4.9 release, as well as the 5.x
release which drops old cruft and 1.8 support.  I've decided
to use 4.9 instead of 4.8.4 since this does change the feature
set slightly.

Eric Wong (2):
      tee_input: support for Rack::TempfileReaper middleware
      support TempfileReaper in deployment and development envs

 lib/unicorn.rb              |  2 ++
 lib/unicorn/tee_input.rb    |  9 ++++++++-
 lib/unicorn/tmpio.rb        |  3 +++
 test/unit/test_tee_input.rb | 10 ++++++++++
 4 files changed, 23 insertions(+), 1 deletion(-)

Jason Hines | 24 Apr 02:04 2015

Will restarting a Unicorn process (via HUP signal) wait for worker threads?

In my application, I have some background jobs which are executed in
separate threads.  (using SuckerPunch/Celluloid framework)

If I send a HUP signal to the Unicorn master, the application is reloaded
and gracefully restarts the workers.    But, if those workers have child
threads, does Unicorn wait for those threads to finish before the restart
or are they terminated immediately?

Thanks in advance for any light shed here.

Jérémy Lecour | 23 Apr 11:05 2015
Picon

Unicorn, environment variables, start and reload

Hi,

For some time I've been using Unicorn to serve Rails applications.

I've been increasingly relying on environment variables to set various
password and configuration bits outside of the application's code.

For that I've been using the "dotenv" gem that load a `.env` file into
the ENV hash. It's great and really useful in development mode, but
it's not a best practice to use it in production. Here is what Brandon
Keepers (maintainer of Dotenv) says about this :

> One of the reasons I don't advocate for using dotenv in production is because it loads the environment
variables within the ruby process, which makes it very difficult to track which variables were
previously set and which were loaded by dotenv. If you set these variables in the environment of your
server (/etc/environment, /etc/profile, etc) then unicorn reloading will just work.

So I was wondering what would be the correct way to have environment
variables available in the Ruby process, up-to-date when the Unicorn
process is started and reloaded too (with USR2).

And there is also the case of various shell initializations
(login/non-login, interactive/non-interactive) and supervisors like
Monit that strip the environment to a minumum before executing the
start/stop commands.

I understand that I can do something like this on start :

    $ source ~/my/app/.env && unicorn [...]

(Continue reading)

Eric Wong | 22 Apr 21:02 2015

unicorn 4.8.x-stable branch pushed to git

Only backporting documentation + build/test system fixes, but I'll
probably apply and push the TeeInput patch in:
http://bogomips.org/unicorn-public/m/20150422183808.GA5277 <at> dcvr.yhbt.net.txt

The following changes since commit 7087bb7ed5a1b9d9f24069cb92707d086668b6dc:

  unicorn 4.8.3 - the end of an era (2014-05-07 07:49:19 +0000)

are available in the git repository at:

  git://bogomips.org/unicorn 4.8.x-stable

for you to fetch changes up to 548e1e67d314f6ebd17df37ece0ee20632462f6f:

  doc: document UNICORN_FD in manpage (2015-04-22 18:57:39 +0000)

----------------------------------------------------------------
Eric Wong (19):
      ISSUES: update with mailing list subscription
      FAQ: add entry for Rails autoflush_log
      dev: remove isolate dependency
      unicorn.gemspec: depend on test-unit 3.0
      remove RubyForge and Freecode references
      remove mongrel.rubyforge.org references
      examples: add run_once to before_fork hook example
      t/t0002-parser-error.sh: relax test for rack 1.6.0
      switch docs + website to olddoc
      README: clarify/reduce references to unicorn_rails
      gemspec: fixup olddoc migration
      GNUmakefile: fix clean gem build + reduce build cruft
(Continue reading)

Mulvaney, Mike | 22 Apr 18:42 2015

TeeInput leaks file handles/space

When I upload large files to my unicorn process, TeeInput is streaming them through a  <at> tmp instance
variable which writes to /tmp/0.123456789 (some random number).  The file is immediately deleted but the
file handle is not closed, so the files are not really deleted by the file system.

The files are eventually deleted when GC runs, but before GC runs they continue to take up space.  You can see
this in lsof output, for example:

ruby       2783   webuser  23u      REG               0,17  6128086    3556917 /tmp/0.04249158625633187 (deleted)

This can cause problems if you have big files and a small /tmp, such as a tmpfs disk mounted in ram.  If someone
sends in several 100MB files, you could easily get 2-3 open files for each of 6 unicorn processes, which
would take up 1200MB of disk space until GC decides to run.

I looked into fixing this but it doesn't look easy.  I can reach into the TeeInput variable and close out the
 <at> tmp instance variable in my application, and that does fix the problem.  But obviously that is not a good
solution.  I think there would have to be some kind of "close" method on http_request that would close out
all the open resources such as these files.

-Mike

Gabe Martin-Dempesy | 8 Apr 18:22 2015

Re: nginx reverse proxy getting ECONNRESET

Follow up on the resolution.

Data was being left in the socket, although it wasn’t immediately obvious why. The affected end point
received gzipped POST bodies, and would process this somewhat like this:

    uncompressed_body = StringIO.new(Zlib::GzipReader.new(request.body).read)
    process(uncompressed_body)

At the end of each request, all of the uncompressed data had been consumed and `uncompressed_body.eof?`
would always be true. However, GzipReader#close wasn’t explicitly called, causing anywhere between
1 and 7 trailing bytes of the 8 byte gzip footer to remain unread in request.body (rack.input) in a small
portion of requests. Rewriting this to explicitly close the GzipReader forces it to read and validate the
footer, and leaves us with a completely consumed request.body.

Thanks for your help identifying the root cause.

Eric Wong | 8 Apr 01:22 2015

possible errors reading mailing list archives

If you hit any errors or truncated/corrupted responses when reading ML
archives at http://bogomips.org/unicorn-public/ it's probably because of
a bug in the new rack.hijack-based proxy in front of the prefork Apache2
instance hosting.

Drop me (or this list) a plain-text mail with the URL(s) and any info
about your client/connection that's broken for you

Previously, it was yahns using a synchronous (Rack, no-hijack) proxy
from yahns in the extras/proxy_pass.rb file[2]

before:
  yahns:extras/proxy_pass.rb -> varnish -> mpm_prefork+mod_perl

after:
  yahns:lib/yahns/proxy_pass.rb -> varnish -> mpm_prefork+mod_perl

yahns mailing list archives with all the relevant commits are here:
http://yhbt.net/yahns-public/

In case the HTTP portion of the archives get hosed, archives of the
mailing lists are still git clonable with ssoma[1] (or just using
"git clone")

	git://yhbt.net/yahns-public
	git://bogomips.org/unicorn-public

In case the server is gets completely hosed by the yahns proxy_pass
changes, archives are still readable at:

(Continue reading)

Eric Wong | 7 Apr 09:56 2015

[PATCH] favor more string literals for cold call sites

Literal regexps cost over 450 bytes of memory per-site and
unnecessary use of them costs memory in places where raw execution
speed does not matter.  Nowadays, we can rely on String#end_with?
(introduced in 1.8.7) for improved readability, too.
---
 bin/unicorn                  | 2 +-
 bin/unicorn_rails            | 2 +-
 lib/unicorn/configurator.rb  | 4 ++--
 lib/unicorn/http_server.rb   | 2 +-
 lib/unicorn/socket_helper.rb | 2 +-
 5 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/bin/unicorn b/bin/unicorn
index c272e43..3c5e5cb 100755
--- a/bin/unicorn
+++ b/bin/unicorn
 <at>  <at>  -29,7 +29,7  <at>  <at>  op = OptionParser.new("", 24, '  ') do |opts|

   opts.on("-I", "--include PATH",
           "specify $LOAD_PATH (may be used more than once)") do |path|
-    $LOAD_PATH.unshift(*path.split(/:/))
+    $LOAD_PATH.unshift(*path.split(':'))
   end

   opts.on("-r", "--require LIBRARY",
diff --git a/bin/unicorn_rails b/bin/unicorn_rails
index b080846..ea4f822 100755
--- a/bin/unicorn_rails
+++ b/bin/unicorn_rails
 <at>  <at>  -30,7 +30,7  <at>  <at>  op = OptionParser.new("", 24, '  ') do |opts|
(Continue reading)

Russell Jennings | 27 Mar 15:32 2015
Picon

$USER and $HOME shell variables not set

Hello,

I am running into some issues with these variables being set - since I am spawning a script from a unicorn
worker (via a rails controller) I figured I’d ask here. 

Here is the stackoverflow with the full background:
http://stackoverflow.com/questions/29233181/why-is-envhome-nil-in-my-rake-task

in short: does unicorn have anything to do with $HOME or $USER not being defined? From what I can tell, that
its unicorn is the only thing thats different versus running the same ruby via a rails console (which does
indeed set those shell variables correctly)

in no mans land, so any hep or insight would be greatly appreciated. 

Thanks,
Russell

Michael Fischer | 24 Mar 23:43 2015

nginx reverse proxy getting ECONNRESET

We have an nginx 1.6.2 proxy in front of a Unicorn 4.8.3 server that
is frequently reporting the following error:

2015/03/24 01:46:01 [error] 11217#0: *872231 readv() failed (104:
Connection reset by peer) while reading upstream

The interesting things are:

1) The upstream is a Unix domain socket (to which Unicorn is bound)
2) Unicorn isn't reporting that a child died in the error log (and I
verified their lifetimes with ps(1))

Any hints as to what we should look for?

Thanks,

--Michael

胡明 | 24 Mar 16:28 2015
Picon

Is it possible to create a thread in Rails subscribe to Redis message channel?

Hi there

I have one question about related to thread in Unicorn; I have asked the
quesiton on stackoverflow.com but no one answers:

http://stackoverflow.com/questions/29180275/is-it-possible-to-create-a-thread-in-rails-subscribe-to-redis-message-channel

I also pasted the question in this email, hope that someone could help me
with this:

==============================================

I am trying to create a thread in Rails to subscribe a message channel of
Redis. Is there a way to do this? I am using unicorn.

I have tried to do this in the unicorn configuration like this:

after_fork do |server, worker|

  Thread.new do
    begin
      $redis.subscribe(:one, :two) do |on|
        on.subscribe do |channel, subscriptions|
          puts "Subscribed to ##{channel} (#{subscriptions} subscriptions)"
        end
        on.message do |channel, message|
          puts "##{channel}: #{message}"
          $redis.unsubscribe if message == "exit"
        end
        on.unsubscribe do |channel, subscriptions|
(Continue reading)


Gmane