headers
Ambalu, Robert | 25 May 16:00
Picon
Favicon

Pub/Sub over PGM with many topics

Hi, I’m new to zeroMQ and am trying to understand how pub.sub over multicast would work in my scenario.

I want to have one publisher which publishes say 10k different topics.  I can have N subscriber processes, each of which can subscribe to multiple topics

So, use case example.  Say my publisher is publishing market data for 10k different instruments, my subscriber app only handles 100 stocks so it only want to get messages for those 100 stocks.  In addition, I don’t want to have my app do any type of topic resolution, but if possible let zeromq handle the topic multiplexing. 

So I was thinking on the subscriber side, for every topic I have it looks like I would have to create a separate SUB socket to the same pgm multicast channel, but set with a different SUBSCRIBE topic.  Then I would have to poll on all sockets ( essentially, all topics ) and then dish out to the appropriate pollitem ( which in this case would be my object, so say data for AAPL would be subscribed to my an AAPL pollitem handler which knows what it is / no lookup necessary. ( This is all somewhat trying to mimick what Rendezvous can do )

 

A few questions here:

- first of all, is this at all proper?

- if I do need to create a socket per topic, and I subscribe to lets say 1000 topics, does that mean I need 1000 sockets, and then for every single poll() iterate through all 1000 sockets? That doesn’t seem right / inefficient, so im hoping I’m thinking about this incorrectly

- whats the deal with multipart messages and topics? They seem very intertwined and a bit confusing.  On the publisher side it just sends a multipart message to send a specific topic? But on the receiver side it has to subscribe to said topic in sockopts, and then process the message as a multipart message ( again, I would prefer not to even process the multipart message but just get a regular message for the bound topic )

 

Hope these questions are clear, thanks in advance

 

- Rob

This email is confidential and subject to important disclaimers and conditions including on offers for the purchase or sale of securities, accuracy and completeness of information, viruses, confidentiality, legal privilege, and legal entity disclaimers, available at http://www.jpmorgan.com/pages/disclosures/email. 

_______________________________________________
zeromq-dev mailing list
zeromq-dev <at> lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Permalink | Reply | 0 comments |
headers
Sam Zaroug | 25 May 10:01
Picon
Favicon

PUSH/PULL not working in this example

Hi Folks,


I ve the following client server executables that arent working on my machine (attached)


please, note: 
- replace boost  calls with std::cout if you dont use boost in your dev env.


the reciever seems to wait forever, and the sender exit without waiting for the message being processed.
if you uncomment the _sleep(1) on the sending side things works just fine.


I wondering if there is any logical explanation to this simple senario?




Kind Regards


Brownviper1966






Attachment (send.cpp): application/octet-stream, 842 bytes
Attachment (receive.cpp): application/octet-stream, 1015 bytes
_______________________________________________
zeromq-dev mailing list
zeromq-dev <at> lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Permalink | Reply | 0 comments |
headers
Jonathan.Meran | 25 May 16:24

3.1 stability status

Hello,

I plan to bring ZeroMQ into my codebase to start testing out its pub/sub capabilities. I had brought in ZeroMQ 2.1 in a former project and found it a bit painful to resolve the uuid dependency on the embedded Linux platform we use. I hear 3.1 no longer contains that dependency and am interested in jumping to that revision if it is fairly stable. There are also some other features in 3.1 that I could benefit from. Is there a timeframe as to when 3.1 will be considered the new “stable release”? Are there any big issues with it today?

 

/Jon

 

_______________________________________________
zeromq-dev mailing list
zeromq-dev <at> lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Permalink | Reply | 2 comments |
headers
Tarek Ziadé | 24 May 14:17
Favicon
Gravatar

several binds on the same ipc

Hey

I am trying to solve an issue I have with a broker that opens a zmq 
router socket and just relay calls to workers via a socket dealer.

The problem is that I don't know how to prevent a second broker to bind 
itself to the same ipc.

Consider the following scripts where I isolate the issue:

The server: http://tarek.pastebin.mozilla.org/1649175
The client: http://tarek.pastebin.mozilla.org/1649176

If I run them they happily interact. Now I can run a second server. It 
will bind the socket but won't do anything or even error out.

If I stop the first server, the client will simply lock, as it seems to 
have a sticky connection to the first server - and won't communicate
with the second server until I restart it.

As far as I understand, we can only bind() *once*, which make sense.

So my question is - shouldn't the second attempt to bind() the socket 
raise an error ? maybe this is related to the python binding ?

If not, how can I know if there's already a socket bound to the ipc ?

Cheers
Tarek
Permalink | Reply | 7 comments |
headers
Chick Corea | 24 May 05:02
Picon

exhausting limited FDs

I have a non-hypothetical concern re: FD usage with ZMQ.  Admittedly,
I am a typical programmer having implemented most inter-proc comm via
bare sockets.

As I understand the general use-model, a program would would use
multiple TCP ports for the various message-types to be sent between
servers and clients, PUB/SUB, REQ/REP, PUSH/PULL.  That being the
case, comm between two procs, a server could easily use 2 or more
sockets for each client.  If a proc is limited to 5k FDs then the
number of clients simultaneously connected is limited to less than
2.5k before it has to start managing those FDs with some kind of LRU
algorith with reconnects as necessary.

That seems to be the inevitable outcome of not being able to map an
endpoint to multiple sockets.  Is that correct?

What's the solution?   Is that actually acceptable?

Thank you.

FWIW, ZMQ seems very promising to use in my current project.  I have
two key concerns re: it; this is one of them.

CZC
Permalink | Reply | 3 comments |
headers
Pieter Hintjens | 23 May 18:11

Downloads site - removed historic folder

Hi all,

We used to move old versions of 0MQ tarballs into /historic.

People rightly complained because this kept breaking things for anyone
linking to a tarball.

I've killed that folder and all past and present releases of 0MQ and
related projects are now simply in the main downloads directory at
http://download.zeromq.org/

-Pieter

Ps. if you were linking to a tarball in /historic sorry, you'll have
one last fix to make.
Permalink | Reply | 0 comments |
headers
Boris Gulay | 23 May 16:20
Picon

ZeroMQ for Android

I've successfully compiled and tested libzmq v2.2 for Android using
latest ndk (r8). I've used static linking for libuuid and libzmq, so I
have only one native file to copy on the device: libjzmq. I believe it;s
better than compile al libraries to .so because on android all such
files are private to the application and separate files just slows down
startup of the application.

Should I write new description of the build process on android page?


_______________________________________________
zeromq-dev mailing list
zeromq-dev <at> lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Permalink | Reply | 1 comment |
headers
Pieter Hintjens | 22 May 22:58

June meetup / drinkup in Bay Are?

Hi all,

Any interest in a meetup in the Bay Area on the 9th of June?

We have office space downtown SF.

-Pieter
Permalink | Reply | 3 comments |
headers
Marco Trapanese | 22 May 20:11
Picon

Code-style question

Hello,

currently I'm using zmq on several projects to control remote 
applications. In fact, I send 'commands' rather 'data'.

The architecture of the whole software is pretty simple. There are the 
basic functions that do the hard job (my own API). Let's call 'em 
func1(), func2() ...

Then I wrote a raw protocol to call these functions through the zmq 
connection. After a message is received, I check the first part to 
decode the command and the followings to get the parameters (if any). 
Something like this:

void parseMessage() {
	// receive the message, get cmd, argc and argv[]

	switch (cmd) {
	case CMD1:
		func1();
		break;

	case CMD2:
		if (argc == 1) func2(argv[0]);
		break;
	}
}

I wonder how do you implement such an interface. Do you use a method 
like this or there is something more flexible/elegant?

Thanks
Marco
Permalink | Reply | 9 comments |
headers
Victor Perron | 22 May 04:56

Android configuration scripts

Hello on this list,

At my work I had to generate an Android-compatible .so that could be loaded directly in an APK, and use Jzmq.

The result, a collection of harmless configuration scripts, is presented there:

http://iso3103.blogspot.jp/2012/05/java-zeromq-for-android.html
and associated github
https://github.com/vperron/android-jzeromq

I've seen quite a lot of questions about this matter on the web, so maybe that could be of any help.

I have a few questions though :
- Do you zmq devs think it is an acceptable way to go ?
- Do you, people interested with zeromq on Android, meet issues using this ?
- Wouldn't it be more useful converted in some other way, for e.g a post on the :build wiki or a 'target' in the distribution ?
- Am I totally wrong and there is a straightforward way ?

Thanks a lot ,
best regards


--
Victor

_______________________________________________
zeromq-dev mailing list
zeromq-dev <at> lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Permalink | Reply | 1 comment |
headers
Steven McCoy | 22 May 03:54
Picon
Favicon

std::numeric_limits <size_t> invalid

Not supported with MSVC, needs an alternative method.  C99 route is an alternative:  SIZE_MAX.  I'm sure something old hates that.


_______________________________________________
zeromq-dev mailing list
zeromq-dev <at> lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Permalink | Reply | 3 comments |

Gmane