Daniel Krikun | 1 May 11:51 2012
Picon

zmq_poll loop break; performance

Hello,

I have the following code to dispatch incoming messages via inproc transport in one thread, and to stop the dispatcher from another (main) thread:


void dispatch()
{
      zmq::socket_t  control_socket_ ( _global_context, ZMQ_PAIR );
      control_socket_ .connect( "inproc://control" );

     zmq::pollitem_t items[] = {
           { data_socket_, 0, ZMQ_POLLIN, 0 },
           {  control_socket_, 0 , ZMQ_POLLIN, 0 }
      }; 

      while(1)
      {
               zmq::poll( items, 2)
               if(items[0].revents & ZMQ_POLLIN)
               {
                             // ... dispatch incoming messages ..
               }
               if(items[1].revents & ZMQ_POLLIN)
                     break;
       }
}


In another (main) thread, I send an empty message to "inpoc://control" when I want to stop the dispatch.
The problem that this stopping is somehow very slow.
Why could be that?      

I am using zeromq-2.1.11 on Windows XP SP3, Visual Studio 2008.

Thanks,
--
Daniel Krikun

_______________________________________________
zeromq-dev mailing list
zeromq-dev <at> lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Chuck Remes | 1 May 16:12 2012

Re: zmq_poll loop break; performance


On May 1, 2012, at 4:51 AM, Daniel Krikun wrote:

> Hello,
> 
> I have the following code to dispatch incoming messages via inproc transport in one thread, and to stop the
dispatcher from another (main) thread:
> 
> 
> void dispatch()
> {
>       zmq::socket_t  control_socket_ ( _global_context, ZMQ_PAIR );
>       control_socket_ .connect( "inproc://control" );
> 
>      zmq::pollitem_t items[] = {
>            { data_socket_, 0, ZMQ_POLLIN, 0 },
>            {  control_socket_, 0 , ZMQ_POLLIN, 0 }
>       }; 
> 
>       while(1)
>       {
>                zmq::poll( items, 2)
>                if(items[0].revents & ZMQ_POLLIN)
>                {
>                              // ... dispatch incoming messages ..
>                }
>                if(items[1].revents & ZMQ_POLLIN)
>                      break;
>        }
> }
> 
> 
> In another (main) thread, I send an empty message to "inpoc://control" when I want to stop the dispatch.
> The problem that this stopping is somehow very slow.
> Why could be that?      
> 
> I am using zeromq-2.1.11 on Windows XP SP3, Visual Studio 2008.

Can you define what you mean by "slow?" Is it 10 milliseconds, 100 milliseconds, 1000 milliseconds, 10_000 milliseconds?

Also, is it slow *every* time?

Is it dependent upon how long the program has been running OR how many "data" messages you have dispatched?

cr
Daniel Krikun | 1 May 18:20 2012
Picon

Re: zmq_poll loop break; performance

About 2-3 seconds delay. Don't know whether it is dependent on number of message dispatched, I'd tried in a pilot only, which was about 20 messages.

On Tue, May 1, 2012 at 5:12 PM, Chuck Remes <lists <at> chuckremes.com> wrote:

On May 1, 2012, at 4:51 AM, Daniel Krikun wrote:

> Hello,
>
> I have the following code to dispatch incoming messages via inproc transport in one thread, and to stop the dispatcher from another (main) thread:
>
>
> void dispatch()
> {
>       zmq::socket_t  control_socket_ ( _global_context, ZMQ_PAIR );
>       control_socket_ .connect( "inproc://control" );
>
>      zmq::pollitem_t items[] = {
>            { data_socket_, 0, ZMQ_POLLIN, 0 },
>            {  control_socket_, 0 , ZMQ_POLLIN, 0 }
>       };
>
>       while(1)
>       {
>                zmq::poll( items, 2)
>                if(items[0].revents & ZMQ_POLLIN)
>                {
>                              // ... dispatch incoming messages ..
>                }
>                if(items[1].revents & ZMQ_POLLIN)
>                      break;
>        }
> }
>
>
> In another (main) thread, I send an empty message to "inpoc://control" when I want to stop the dispatch.
> The problem that this stopping is somehow very slow.
> Why could be that?
>
> I am using zeromq-2.1.11 on Windows XP SP3, Visual Studio 2008.

Can you define what you mean by "slow?" Is it 10 milliseconds, 100 milliseconds, 1000 milliseconds, 10_000 milliseconds?

Also, is it slow *every* time?

Is it dependent upon how long the program has been running OR how many "data" messages you have dispatched?

cr


_______________________________________________
zeromq-dev mailing list
zeromq-dev <at> lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev



--
Daniel Krikun

_______________________________________________
zeromq-dev mailing list
zeromq-dev <at> lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Ian Barber | 1 May 18:27 2012
Picon

Re: zmq_poll loop break; performance



On Tue, May 1, 2012 at 5:20 PM, Daniel Krikun <krikun.daniel <at> gmail.com> wrote:
About 2-3 seconds delay. Don't know whether it is dependent on number of message dispatched, I'd tried in a pilot only, which was about 20 messages.



Can you tell where it's being slow - is it happening before the break, or in the shutdown of a thread/process?

Ian
_______________________________________________
zeromq-dev mailing list
zeromq-dev <at> lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
虞红伟 | 2 May 04:37 2012

Dealer and router problems with "inproc" and "tcp"

Hi all,

         I have encountered two problems with zmq_2.1.10, VS2008, windows server 2008R2:

1.       Dealer<->Router with “inproc”. We send messages to dealers with router using corresponding identity firstly, then we closed the dealers, and did not close the router, we found that the handles (windows handle) used by dealers did not released until we closed the router. But we HAVE TO keep the router alive until program exit.

2.       Dealer<->Router with ”tcp”. We initialize the io thread numbers = 2. We use dealer and router send messages to each other, and we get a zmq_assert ( zmq_assert (sessions.empty ()) ) sometimes when we close the router. But we initialize the io thread numbers = 1, the assert never appears.

 

I search the mail list and did not find the answer, is there anyone encountered the same problems, and any ideas?

 

Thanks!

 

 

 

David.

_______________________________________________
zeromq-dev mailing list
zeromq-dev <at> lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Daniel Krikun | 2 May 05:21 2012
Picon

Re: zmq_poll loop break; performance

In the shutdown of a thread/process. The dispatching itself is fast.

On Tue, May 1, 2012 at 7:27 PM, Ian Barber <ian.barber <at> gmail.com> wrote:


On Tue, May 1, 2012 at 5:20 PM, Daniel Krikun <krikun.daniel <at> gmail.com> wrote:
About 2-3 seconds delay. Don't know whether it is dependent on number of message dispatched, I'd tried in a pilot only, which was about 20 messages.



Can you tell where it's being slow - is it happening before the break, or in the shutdown of a thread/process?

Ian

_______________________________________________
zeromq-dev mailing list
zeromq-dev <at> lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev




--
Daniel Krikun

_______________________________________________
zeromq-dev mailing list
zeromq-dev <at> lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Pieter Hintjens | 2 May 06:11 2012

Re: Dealer and router problems with "inproc" and "tcp"

On Wed, May 2, 2012 at 4:37 AM, 虞红伟 <yuhongweiyf <at> hikvision.com> wrote:

> 1.       Dealer<->Router with “inproc”. We send messages to dealers with
> router using corresponding identity firstly, then we closed the dealers, and
> did not close the router, we found that the handles (windows handle) used by
> dealers did not released until we closed the router. But we HAVE TO keep the
> router alive until program exit.

Sounds like normal behaviour. If there is a way to keep the dealer
sockets open, do that.

> 2.       Dealer<->Router with ”tcp”. We initialize the io thread numbers =
> 2. We use dealer and router send messages to each other, and we get a
> zmq_assert ( zmq_assert (sessions.empty ()) ) sometimes when we close the
> router. But we initialize the io thread numbers = 1, the assert never
> appears.

Can you make a minimal reproducible test case?

Then create an issue, and we can fix it.

Thanks
Pieter
_______________________________________________
zeromq-dev mailing list
zeromq-dev <at> lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Thomas Fee | 2 May 07:18 2012
Picon

Re: I've written a completely new C# API for ZeroMQ

Hey Alex, sounds great but no way I can get it to work. It's probably because 
I'm a newb at this C# stuff. Would appreciate a quick look at this, a cut and 
paste of your code into a program "shell". At this point, I'm just trying to get 
it to compile, never mind have it work.

What I did was copy the DLL (that came with your distro) into my C# project 
directory. Then in Solution Explorer, I added your DLL as a reference (using 
"Add Reference", then Browse tab).

The compile error I get is

'ZeroMQ.ZmqSocket.Bind(string)' is inaccessible due to its protection level

Ditto for .Connect(string).

What does this mean? Would be great if you could write a beginner's tutorial. :-
)

Here is the code:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Data;
using System.Windows.Documents;
using System.Windows.Input;
using System.Windows.Media;
using System.Windows.Media.Imaging;
using System.Windows.Navigation;
using System.Windows.Shapes;
using ZeroMQ;

namespace WpfApplication1
{
    public partial class MainWindow : Window
    {
        public MainWindow()
        {
            InitializeComponent();

            var publisher = new ZmqPublishSocket
            {
                Identity = Guid.NewGuid().ToByteArray(),
                RecoverySeconds = 10
            };

            publisher.Bind(address: "tcp://127.0.0.1:9292");
            var subscriber = new ZmqSubscribeSocket();
            subscriber.Connect(address: "tcp://127.0.0.1:9292");
            subscriber.Subscribe(prefix: ""); // subscribe to all messages
            subscriber.OnReceive += () =>
            {
                String message;
                subscriber.Receive(out message, nonblocking: true);
                Console.WriteLine(message);
            };

            publisher.Send("Hello world!");
            return 0;
        }
    }
}
Gerhard Lipp | 2 May 09:11 2012

Re: missing events ZMQ_FD / ZMQ_EVENTS

hello paul!

i dont understand the background of your approach. why should the src
fd's io handler check the dst's events (and vice versa)?
even if this worked in this scenario, wouldn't it be a coincidence?
well, at least it is better than busy waiting / polling ...
regards

On Fri, Apr 27, 2012 at 9:10 PM, Paul Colomiets <paul <at> colomiets.name> wrote:
> Hi Gerhard,
>
> On Fri, Apr 27, 2012 at 2:10 PM, Gerhard Lipp <gelipp <at> googlemail.com> wrote:
>> Ok, so i must always check if there are more events to process before
>> returning from the io handler (frankly I don't understand the
>> explanation). A short test still shows the lock explained earlier:
>
> Try the following:
>
> local zmq = require'zmq'
> local ev = require'ev'
> local c = zmq.init(1)
> local xreq = c:socket(zmq.XREQ)
> xreq:bind('tcp://127.0.0.1:13333')
> local xrep = c:socket(zmq.XREP)
> xrep:bind('tcp://127.0.0.1:13334')
>
> local is_readable =
>  function(sock)
>     local events = sock:getopt(zmq.EVENTS)
>     return events == zmq.POLLIN or events == (zmq.POLLIN + zmq.POLLOUT)
>  end
>
> local forward_io =
>  function(src,dst)
>     return ev.IO.new(
>        function(loop,io) -- called whenever src:getopt(zmq.FD) becomes readable
>            while is_readable(src) or is_readable(dst) do
>               if is_readable(src) do
>                  repeat
>                     local data = assert(src:recv(zmq.NOBLOCK))
>                     local more = src:getopt(zmq.RCVMORE) > 0
>                     dst:send(data,more and zmq.SNDMORE or 0)
>                  until not more
>               end
>               if is_readable(dst) do
>                  repeat
>                     local data = assert(dst:recv(zmq.NOBLOCK))
>                     local more = dst:getopt(zmq.RCVMORE) > 0
>                     src:send(data,more and zmq.SNDMORE or 0)
>                  until not more
>               end
>            end
>        end,
>        src:getopt(zmq.FD),
>        ev.READ
>     )
>  end
> local xrep_io = forward_io(xrep,xreq)
> local xreq_io = forward_io(xreq,xrep)
> xreq_io:start(ev.Loop.default)
> xrep_io:start(ev.Loop.default)
> ev.Loop.default:loop()
>
> If this works. You can optimize (and clarify) it more.
>
> --
> Paul
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev <at> lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Gerhard Lipp | 2 May 10:26 2012

Re: missing events ZMQ_FD / ZMQ_EVENTS

btw, using the build in poller just works:

-------
-- xpoller.lua
---------
local zmq = require'zmq'
zmq.poller = require'zmq.poller'
local ev = require'ev'
local c = zmq.init(1)
local xreq = c:socket(zmq.XREQ)
xreq:bind('tcp://127.0.0.1:13333')
local xrep = c:socket(zmq.XREP)
xrep:bind('tcp://127.0.0.1:13334')

local is_readable =
   function(sock)
      local events = sock:getopt(zmq.EVENTS)
      return events == zmq.POLLIN or events == (zmq.POLLIN + zmq.POLLOUT)
   end

local forward =
   function(src,dst)
      while is_readable(src) do
         repeat
            local data = assert(src:recv(zmq.NOBLOCK))
            local more = src:getopt(zmq.RCVMORE) > 0
            dst:send(data,more and zmq.SNDMORE or 0)
         until not more
      end
   end

local xpoller = zmq.poller.new()
xpoller:add(xreq,zmq.POLLIN,
            function()
               forward(xreq,xrep)
            end)

xpoller:add(xrep,zmq.POLLIN,
            function()
               forward(xrep,xreq)
            end)

xpoller:start()

On Mon, Apr 23, 2012 at 2:53 PM, Gerhard Lipp <gelipp <at> googlemail.com> wrote:
> Hello,
>
> I can observe the same behavior as stated here
> (http://lists.zeromq.org/pipermail/zeromq-dev/2011-November/014615.html).
> What I observe is also a XREP/XREQ (ROUTER/DEALER) prob, where the
> XREQ is waiting forever to receive a message (which has been
> definitely sent). When I poll (timer based) the ZMQ_EVENTs, the XREQ
> is readable as expected. I am using libev (select based) for doing IO
> and I am aware of the edge-based trigger behaviour (I am
> reading/forwarding messages until ZMQ_EVENTs does not include the
> ZMQ_POLLIN bit any more).
>
> What is the status of this issue?
> Unfortunately my setup is a bit complicated to share, but i would like
> to help as much as possible.
>
> Regards,
> Gerhard
>
> A libev workaround is to use both EV_READ and EV_WRITE bits, though
> this adds a lot of unnecessary wake ups / callbacks etc.

Gmane