Michal Slaski | 6 Sep 16:32 2007

Long queues with EXIT signals

Hi,
I have the latest 1.68 Yaws running and receiving 5 requests per
second with peaks of 40 requests per second. Some of the Yaws
processes have very long message queues with peak up to 1500. All
messages are exit signals like this: {'EXIT',#Port<0.23517>,normal}.
The initail call of those processes is the yaws_server:acceptor0/2
function and they are waiting in prim_inet:recv0/3 most of the time.

It might be that some of the request that we are getting are not
proper HTTP reqests or that client on the other end does some magic
with opening and closing sockets. I need to investigate that more. But
my question is has anyone seen such long queues with exit signals
before?

Thanks
--

-- 
Michal Slaski
http://www.erlang-consulting.com

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
Claes Wikstrom | 6 Sep 19:29 2007

Re: Long queues with EXIT signals

Michal Slaski wrote:
> Hi,
> I have the latest 1.68 Yaws running and receiving 5 requests per
> second with peaks of 40 requests per second. Some of the Yaws
> processes have very long message queues with peak up to 1500. All
> messages are exit signals like this: {'EXIT',#Port<0.23517>,normal}.
> The initail call of those processes is the yaws_server:acceptor0/2
> function and they are waiting in prim_inet:recv0/3 most of the time.
> 

This basically means that there are a lot of sockets that are
being closed on the client side and Yaws hasn't yet picked up on the
fact that the client sockets are closed.  The emulator should be running
at 100% when you see this ??

The code in prim_inet.erl recv0() is just aching to pick up
that EXIT signal and report the socket as closed.

/klacke

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
Michal Slaski | 7 Sep 11:41 2007

Re: Long queues with EXIT signals

Claes Wikstrom wrote:
> This basically means that there are a lot of sockets that are
> being closed on the client side and Yaws hasn't yet picked up on the
> fact that the client sockets are closed.  The emulator should be  
> running
> at 100% when you see this ??

Actually no. The beam process has peaks of 10% CPU utilization and  
the system is not overloaded at all. After couple of minutes the  
acceptor process dies and is restarted so its mailbox is empty. I  
don't see any error messages.

Michal

--

-- 
Michal Slaski
http://www.erlang-consulting.com

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
Claes Wikstrom | 7 Sep 12:36 2007

Re: Long queues with EXIT signals

Michal Slaski wrote:
> Claes Wikstrom wrote:
>> This basically means that there are a lot of sockets that are
>> being closed on the client side and Yaws hasn't yet picked up on the
>> fact that the client sockets are closed.  The emulator should be running
>> at 100% when you see this ??
> 
> Actually no. The beam process has peaks of 10% CPU utilization and the 
> system is not overloaded at all. After couple of minutes the acceptor 
> process dies and is restarted so its mailbox is empty. I don't see any 
> error messages.
> 
> Michal
> 

Suspicious, as you might know, in yaws there is one process per socket,
that process will sit in a gen_tcp:recv() call, when the socket dies, it
will send an EXIT sugnal and the socket will be reported as down, e.g
gen_tcp:recv() returns {error, closed}

The process will the report back to its top process that it is done.
In order to go forth with this - some debugging is probably required.
What are the HTTP messages getting in ? are there really processes in
prim_inet:recv() that do have an EXIT msg in the queue and the load is
still 10 % - don't think so .......

To debug/inspect these kind of behaviours, I usually use the bt() function in
my user_default.erl

Attached .
(Continue reading)

Michal Slaski | 7 Sep 12:45 2007

Re: Long queues with EXIT signals

Claes Wikstrom wrote:
> To debug/inspect these kind of behaviours, I usually use the bt()  
> function in
> my user_default.erl

Many thanks. Will trace the incoming traffic and the acceptor process  
and see what happens.

Michal

--

-- 
Michal Slaski
http://www.erlang-consulting.com

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
Sergei Golovan | 16 Sep 20:35 2007
Picon

Images in 1.70 and 1.71 are broken

Hi!

Looking at yaws 1.70, I've noticed that images included into its
distribution are broken. Today I've found new 1.71 with fixed images
for wiki. But for other applications they are still broken (chat and
mail).

Cheers!
--

-- 
Sergei Golovan

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
Claes Wikstrom | 16 Sep 21:02 2007

Re: Images in 1.70 and 1.71 are broken

Sergei Golovan wrote:
> Hi!
> 
> Looking at yaws 1.70, I've noticed that images included into its
> distribution are broken. Today I've found new 1.71 with fixed images
> for wiki. But for other applications they are still broken (chat and
> mail).
> 
> Cheers!

Hmmm, I thought I fixed all the images. I'll check again. And
yes - 1.70 did indeed have broken .png images in e.g. the wiki
application.

It was (is) svn keywords attached to .{gif,png} files

/klacke

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
Mikael Magnusson | 16 Sep 21:23 2007
Picon

Re: Images in 1.70 and 1.71 are broken

Claes Wikstrom wrote:
> Sergei Golovan wrote:
>> Hi!
>>
>> Looking at yaws 1.70, I've noticed that images included into its
>> distribution are broken. Today I've found new 1.71 with fixed images
>> for wiki. But for other applications they are still broken (chat and
>> mail).
>>
>> Cheers!
> 
> 
> Hmmm, I thought I fixed all the images. I'll check again. And
> yes - 1.70 did indeed have broken .png images in e.g. the wiki
> application.
> 
> It was (is) svn keywords attached to .{gif,png} files
> 
> /klacke
> 

You should set svn:mime-type to application/octet-stream for all binary 
files, I think.

Mikael

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
(Continue reading)

Claes Wikstrom | 16 Sep 22:06 2007

Re: Images in 1.70 and 1.71 are broken

Mikael Magnusson wrote:

> You should set svn:mime-type to application/octet-stream for all binary 
> files, I think.
> 

I thought it sufficed to just remove all svn props. Still think that.

/klacke

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
Mikael Magnusson | 16 Sep 22:28 2007
Picon

Re: Images in 1.70 and 1.71 are broken

Claes Wikstrom wrote:
> Mikael Magnusson wrote:
> 
>> You should set svn:mime-type to application/octet-stream for all 
>> binary files, I think.
>>
> 
> I thought it sufficed to just remove all svn props. Still think that.
> 
> /klacke
> 

Yes, but svn:mime-type are also used for other purposes, for example 
when running diff. Are you really interested in textual differences 
between two revisions of for example a png image?

Subversion will also try to merge text files when necessary, which you 
don't want to do on binary files.

http://svnbook.red-bean.com/en/1.2/svn.advanced.props.html

Changes to text files are also included in the commit mail.

Mikael

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
(Continue reading)


Gmane