headers
Abhishek Kona | 1 Mar 2011 04:43
Picon
Gravatar

Re: Thrift vs. Protocol Buffers as serialization formats

Hi

We use Thrift as a thin layer infront of RabbitMQ to provide Amazon SQS 
(acknowledge without keeping a channel open).  like semantics for our 
queue system.
It has been working like a charm for us.
We primarily use in Java and PHP.
Serialization has never been observed as a major bottleneck in our 
application.
So any of PB or Thrift should be good for you. (We use Thrift for its 
good RPC capabilities).

The people at https://github.com/eishay/jvm-serializers/wiki/ have 
profiled lot of available data serialization methods, you can have a look.

-Abhishek Kona

On 28/02/11 10:34 PM, Marek Majkowski wrote:
> On Mon, Feb 28, 2011 at 15:13, SimonT<simonthorogood@...>  wrote:
>> Anyone care to share their experiences with using either of these
>> (specificially, as the 'on-the-wire' format for RabbitMQ messages)? In
>> particular, would be interested to know about:
>>
>> - Ease of managing the (re)generation of message formats for use
>> across multiple platforms (in our case, C# and Python)
>> - Serialisation/Deserialisation performance
>> - Serialized message size
> In my projects the performance of (de)serialization doesn't matter
> that much. But flexibility and simplicity matters a lot.
>
(Continue reading)

Permalink | Reply | 0 comments |
headers
Matthias Radestock | 1 Mar 2011 08:22
Favicon

Re: Losing messages, why?

DOBRO Sergei wrote:
>> In the list_* output you sent, which consumers/queues are in the 
>> "broken" state where their connected clients aren't receiving
>> messages?
>> 
> All the consumers, queues, bindings and connections are present.

Understood, but my question was *which specific* worker(s) (and their
associated consumers/queues) had stopped working at the time you ran
these commands?

> I just can say that when the workers stop receiving messages I
> restart them and everything goes fine. Mysticism :)

If restarting them works fine then the problem is most certainly client 
side.

As I said, you may want to tap the network to check whether messages do 
indeed still get sent to the broken workers.

Regards,

Matthias.
Permalink | Reply | 14 comments |
headers
Rom | 1 Mar 2011 04:55
Picon

Maximum Frame Setting Not Working

I have successfully run a very simple message publisher with my
ConnectionFactory setting below.

                ConnectionFactory factory = new ConnectionFactory();
		factory.setUsername("rabbitmq");
		factory.setPassword("rabbitmq");
		factory.setVirtualHost("/4couchdb");
		factory.setHost("rcgarcia74");
		factory.setPort(ConnectionFactory.DEFAULT_AMQP_PORT);
		factory.setRequestedChannelMax(100);
		factory.setRequestedHeartbeat(300);

But when I start adding in the Maximum Frame size below, any number
that I put in doesn't work. Is there a variable that I need to set at
rabbitmq.conf? If it's in the conf file what's the key/vakue pair?

                factory.setRequestedFrameMax(1);

Here's the exception I'm getting.

java.io.IOException
	at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:121)
	at com.rabbitmq.client.impl.AMQChannel.exnWrappingRpc(AMQChannel.java:
145)
	at com.rabbitmq.client.impl.AMQConnection.start(AMQConnection.java:
311)
	at
com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:
379)
	at
(Continue reading)

Permalink | Reply | 3 comments |
headers
mysurf mail | 1 Mar 2011 10:14
Picon

Re: socket error while using rabbitmq and a NAT (Network address translation)

Hi, 
Thanks for your response. 

I followed your advice and here are my results.
 I see myself in the connection list when I invoke a connection from a computer that gets the exception. I see it with "starting"  status and instantly it disappears. (I use new exchange name and no exchange is created)

I tried to follow your advice about the tracer but i didn't find enough info about the file runjava.bat in 2.3.1 (it exists only in the 1.7.0 package. not the 2.3.1.)

it happens from almost half of the servers. not only the ones with the NAT. 
thanks.

On Mon, Feb 28, 2011 at 5:34 PM, Emile Joubert <emile-mQ7lE4MOPXtWk0Htik3J/w@public.gmane.org> wrote:
Hi,

There is no reason why an AMQP connection won't work over NAT. This is equally for version 1.7.0 and 2.3.1. It is likely that the cause for this error is at the network layer.

Does the broker logfile contain any other relevant messages other than the one about the abrupt disconnection?

Can you get "rabbitmqctl list_connections" on the broker to list the connection from the client at any time?
Are you able to establish an AMQP channel and issue the basicConsume(), or does the error occur earlier during connection establishment?

Do you have any way of checking the state of the connection on the NAT device?

Establishing the AMQP connection via the AMQP Tracer ( http://www.rabbitmq.com/javadoc/com/rabbitmq/tools/Tracer.html ) might also provide useful diagnostic information.



-Emile





On 27/02/11 13:06, mysurf mail wrote:
Hi ,
I am writing a new mail because my last issue was solved .
It is just that on a certain computer my application stopped working.
So it is a new issue and I will explain and paste the exception below.

Basically, I did three things :
1. I upgraded from 1.7.0 to 2.3.1 (installed a new erlang also)
2. I upgraded the client jars
3. changed the protocol to ack = true;

channel.basicConsume(queueName, true, queueingConsumer);

well, that certain computer is behind a firewall, and has a NAT (Network
Address Translation ).
It did work prior to the upgrade with the NAT (mySQL also handles it
pretty well).
with the same ports.
Maybe it is the bidirectional channel (since I added acking? I dont
think so...).

what I can see is that.
when the client tries to connect it writes in the logs

Connection closed abruptly

And now, behold, the exception :

[...]

<div><div dir="ltr">Hi,&nbsp;<div>Thanks for your response.&nbsp;</div>
<div><br></div>
<div>I followed your advice and here are my results.</div>
<div>&nbsp;I see myself in the connection list when I invoke a connection from a computer that gets the exception. I see it with "starting" &nbsp;status and instantly it disappears. (I use new exchange name and no exchange is created)</div>
<div><br></div>
<div>I tried to follow your advice about the tracer but i didn't find enough info about the file runjava.bat in 2.3.1 (it exists only in the 1.7.0 package. not the 2.3.1.)</div>
<div><br></div>
<div>it happens from almost half of the servers. not only the ones with the NAT.&nbsp;</div>
<div>thanks.<br><br><div class="gmail_quote">On Mon, Feb 28, 2011 at 5:34 PM, Emile Joubert <span dir="ltr">&lt;<a href="mailto:emile <at> rabbitmq.com">emile@...</a>&gt;</span> wrote:<br><blockquote class="gmail_quote">
Hi,<br><br>
There is no reason why an AMQP connection won't work over NAT. This is equally for version 1.7.0 and 2.3.1. It is likely that the cause for this error is at the network layer.<br><br>
Does the broker logfile contain any other relevant messages other than the one about the abrupt disconnection?<br><br>
Can you get "rabbitmqctl list_connections" on the broker to list the connection from the client at any time?<br>
Are you able to establish an AMQP channel and issue the basicConsume(), or does the error occur earlier during connection establishment?<br><br>
Do you have any way of checking the state of the connection on the NAT device?<br><br>
Establishing the AMQP connection via the AMQP Tracer ( <a href="http://www.rabbitmq.com/javadoc/com/rabbitmq/tools/Tracer.html" target="_blank">http://www.rabbitmq.com/javadoc/com/rabbitmq/tools/Tracer.html</a> ) might also provide useful diagnostic information.<br>
<br><br><br>
-Emile<div class="im">
<br><br><br><br><br>
On 27/02/11 13:06, mysurf mail wrote:<br><blockquote class="gmail_quote">
Hi ,<br>
I am writing a new mail because my last issue was solved .<br>
It is just that on a certain computer my application stopped working.<br>
So it is a new issue and I will explain and paste the exception below.<br><br>
Basically, I did three things :<br>
1. I upgraded from 1.7.0 to 2.3.1 (installed a new erlang also)<br>
2. I upgraded the client jars<br>
3. changed the protocol to ack = true;<br><br>
channel.basicConsume(queueName, true, queueingConsumer);<br><br>
well, that certain computer is behind a firewall, and has a NAT (Network<br>
Address Translation ).<br>
It did work prior to the upgrade with the NAT (mySQL also handles it<br>
pretty well).<br>
with the same ports.<br>
Maybe it is the bidirectional channel (since I added acking? I dont<br>
think so...).<br><br>
what I can see is that.<br>
when the client tries to connect it writes in the logs<br><br>
Connection closed abruptly<br><br>
And now, behold, the exception :<br>
</blockquote>
<br>
</div>
[...]<br>
</blockquote>
</div>
<br>
</div>
</div></div>
Permalink | Reply | 10 comments |
headers
mysurf mail | 1 Mar 2011 10:15
Picon

Cannot connect to rabbitmq from all computers - including wireshark logs.

I can't connect from all the computers in my production environment. 
Some computers connect, and some do not. 
(In a previous version of rabbit (1.7.0.), they all connected.)

on the non-connecting computers when I call factory.newConnection I wait for a few seconds and then receive the following exception 

Caused by: com.rabbitmq.client.PossibleAuthenticationFailureException: Possibly caused by authentication failure at com.rabbitmq.client.impl.AMQConnection.start(AMQConnection.java:289) at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:383) at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:403) at com.programa17.mmm.modules.stopwatch.util.BasicChannel.<init>(BasicChannel.java:37) ... 28 more Caused by: com.rabbitmq.client.AlreadyClosedException: clean connection shutdown; reason: Attempt to use closed channel at com.rabbitmq.client.impl.AMQChannel.ensureIsOpen(AMQChannel.java:181) at com.rabbitmq.client.impl.AMQChannel.rpc(AMQChannel.java:208) at com.rabbitmq.client.impl.AMQChannel.rpc(AMQChannel.java:194) at com.rabbitmq.client.impl.AMQConnection.start(AMQConnection.java:281) ... 31 more in the wire shark a complete transaction look like this  (I  get the purple only in the successful transaction ) AMQP............ .......copyrightS...$Copyright (C) 2007-2011 VMware, Inc..informationS...5Licensed under the MPL.  See http://www.rabbitmq.com/.platformS... Erlang/OTP.productS....RabbitMQ.versionS....2.3.1....PLAIN AMQPLAIN....en_US......... .......productS....RabbitMQ.platformS....Java.informationS...4Licensed under the MPL. See http://www.rabbitmq.com/.copyrightS...$Copyright (C) 2007-2011 VMware, Inc..versionS....2.3.1.PLAIN.....guest.guest.en_US. ...............................................(./............).................................... .(... toClientsEran.direct..............(...........2. ...toClientsEran.5..............2...toClientsEran.5......... ......9.2.....toClientsEran.5toClientsEran.toClientsEran.5...... ........2...........<.....toClientsEran.5....... ......&.<..!amq.ctag-AKZfeyr+9JOkczDfrq49pw==.......M.<.<!amq.ctag-AKZfeyr+9JOkczDfrq49pw==......... toClientsEran.toClientsEran.5.........<....................{"type":"com.programa17.mmm.modules.stopwatch.util.RabbitMQTest$Start","content":"{\"time\":3449321709892756,\"destinationId\":\"toClientsEran.5\"}"}. ........<.(...toManager.rtk..........<............. .......{"type":"com.programa17.mmm.modules.stopwatch.util.RabbitMQTest$Report","content":"{\"time\":999114088,\"destinationId\":\"toClientsEran.5\"}"}. .......<.P..........

Thanks 
<div><div dir="ltr">I can't connect from all the computers in my production environment.&nbsp;<div>Some computers connect, and some do not.&nbsp;</div>
<div>(In a previous version of rabbit (1.7.0.), they all connected.)</div>
<div><br></div>
<div>on the non-connecting computers when I call factory.newConnection I wait for a few seconds and then receive the following exception&nbsp;</div>
<div><br></div>
<div>Caused by: com.rabbitmq.client.PossibleAuthenticationFailureException: Possibly caused by authentication failure
	at com.rabbitmq.client.impl.AMQConnection.start(AMQConnection.java:289)
	at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:383)
	at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:403)
	at com.programa17.mmm.modules.stopwatch.util.BasicChannel.&lt;init&gt;(BasicChannel.java:37)
	... 28 more
Caused by: com.rabbitmq.client.AlreadyClosedException: clean connection shutdown; reason: Attempt to use closed channel
	at com.rabbitmq.client.impl.AMQChannel.ensureIsOpen(AMQChannel.java:181)
	at com.rabbitmq.client.impl.AMQChannel.rpc(AMQChannel.java:208)
	at com.rabbitmq.client.impl.AMQChannel.rpc(AMQChannel.java:194)
	at com.rabbitmq.client.impl.AMQConnection.start(AMQConnection.java:281)
	... 31 more
<span class="Apple-style-span">in the wire shark a complete transaction look like this &nbsp;(I &nbsp;get the purple only in the successful transaction )</span>
<span class="Apple-style-span">AMQP....</span><span class="Apple-style-span">........</span>
<span class="Apple-style-span">.......copyrightS...$Copyright (C) 2007-2011 VMware, Inc..informationS...5Licensed under the MPL. &nbsp;See <a href="http://www.rabbitmq.com/.platformS.">http://www.rabbitmq.com/.platformS.</a>..</span>
<span class="Apple-style-span">Erlang/OTP.productS....RabbitMQ.versionS....2.3.1....PLAIN AMQPLAIN....en_US.</span><span class="Apple-style-span">........</span>
<span class="Apple-style-span">.......productS....RabbitMQ.platformS....Java.informationS...4Licensed under the MPL. See <a href="http://www.rabbitmq.com/.copyrightS...%24Copyright">http://www.rabbitmq.com/.copyrightS...$Copyright</a> (C) 2007-2011 VMware, Inc..versionS....2.3.1.PLAIN.....guest.guest.en_US.</span>
<span class="Apple-style-span">........</span><span class="Apple-style-span">...........</span><span class="Apple-style-span">........</span><span class="Apple-style-span">...........</span><span class="Apple-style-span">........</span><span class="Apple-style-span">.(./...</span><span class="Apple-style-span">........</span><span class="Apple-style-span">.)..</span><span class="Apple-style-span">..........</span><span class="Apple-style-span">..</span><span class="Apple-style-span">................</span><span class="Apple-style-span">...... .(.</span><span class="Apple-style-span">..</span>
<span class="Apple-style-span">toClientsEran.direct......</span><span class="Apple-style-span">........(...</span><span class="Apple-style-span">........2.</span>
<span class="Apple-style-span">...toClientsEran.5......</span><span class="Apple-style-span">........2...toClientsEran.5.........</span>
<span class="Apple-style-span">......9.2.....toClientsEran.5</span><span class="Apple-style-span">toClientsEran.toClientsEran.5......</span>
<span class="Apple-style-span">........2...</span><span class="Apple-style-span">........&lt;.....toClientsEran.5.......</span>
<span class="Apple-style-span">......&amp;.&lt;..!amq.ctag-AKZfeyr+9JOkczDfrq49pw==.</span><span class="Apple-style-span">......M.&lt;.&lt;!amq.ctag-AKZfeyr+9JOkczDfrq49pw==.........</span>
<span class="Apple-style-span">toClientsEran.toClientsEran.5.........&lt;....................{"type":"com.programa17.mmm.modules.stopwatch.util.RabbitMQTest$Start","content":"{\"time\":3449321709892756,\"destinationId\":\"toClientsEran.5\"}"}.</span>
<span class="Apple-style-span">........&lt;.(...toManager.rtk..</span><span class="Apple-style-span">........&lt;.............</span>
<span class="Apple-style-span">.......{"type":"com.programa17.mmm.modules.stopwatch.util.RabbitMQTest$Report","content":"{\"time\":999114088,\"destinationId\":\"toClientsEran.5\"}"}.</span>
<span class="Apple-style-span">......</span><span class="Apple-style-span">.&lt;.P..........</span>
<div><br></div>
<span class="Apple-style-span">Thanks</span>
</div>
</div></div>
Permalink | Reply | 0 comments |
headers
Emile Joubert | 1 Mar 2011 10:36
Favicon

Re: Maximum Frame Setting Not Working

Hi Rom,

On 01/03/11 03:55, Rom wrote:
> I have successfully run a very simple message publisher with my
> ConnectionFactory setting below.
>
> ConnectionFactory factory = new ConnectionFactory();
> factory.setUsername("rabbitmq"); factory.setPassword("rabbitmq");
> factory.setVirtualHost("/4couchdb"); factory.setHost("rcgarcia74");
> factory.setPort(ConnectionFactory.DEFAULT_AMQP_PORT);
> factory.setRequestedChannelMax(100);
> factory.setRequestedHeartbeat(300);
>
> But when I start adding in the Maximum Frame size below, any number
> that I put in doesn't work. Is there a variable that I need to set
> at rabbitmq.conf? If it's in the conf file what's the key/vakue
> pair?
>
> factory.setRequestedFrameMax(1);

AMQP specifies a minimum frame size of 4kb. You should see this in the 
broker logfile after that attempt:

   amqp_error,not_allowed,"frame_max=1 < 4096 min size"

The solution is to use values of 4096 or higher.

What are you trying to accomplish by changing the default frame size 
(128kb)?

Regards

Emile
Permalink | Reply | 2 comments |
headers
DOBRO Sergei | 1 Mar 2011 10:59
Picon
Favicon

Re: Losing messages, why?

01.03.2011, 10:22, "Matthias Radestock" <matthias <at> rabbitmq.com>:
> DOBRO Sergei wrote:
>
>>>  In the list_* output you sent, which consumers/queues are in the
>>>  "broken" state where their connected clients aren't receiving
>>>  messages?
>>  All the consumers, queues, bindings and connections are present.
>
> Understood, but my question was *which specific* worker(s) (and their
> associated consumers/queues) had stopped working at the time you ran
> these commands?
>

O.k., let me explain what is exactly going on. 
As I said, each worker declares a queue with its own routing key and starts consuming from that queue. All
their queues are bound to the same exchange of direct type.
worker1 -> queue1 -> ex.direct (routing_key1),
worker2 -> queue2 -> ex.direct (routing_key2),
......
worker6 -> queue6 -> ex.direct (routing_key6),

There are six workers. Other clients publish messages to exchange "ex.direct" with the routing key which
is determined randomly from the values of array (routing_key1, routing_key2, ...., routing_key6).
So, the workers receive messages randomly too. That's why I can not tell you which worker(s) stops
receiving messages.

Now let me tell you why I'm using this stupid method :). Why "stupid"? Because it's a random (not
round-robin) dispatching.
But, in my case it is more stable way. Yes, I have already tried to run workers consuming messages from the
same queue, but the issue occured more often. So, having round-robin pattern (one queue) all the workers
stop receiving messages at once.

>
> If restarting them works fine then the problem is most certainly client
> side.
>
> As I said, you may want to tap the network to check whether messages do
> indeed still get sent to the broken workers.
>
RabbitMQ and the workers are running on the same machine. Each connection's IP is 127.0.0.1. Can you give me
some advice? Which software to use to access the data flowing across the connection RabbitMQ<->worker?

Thanks.

_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss <at> lists.rabbitmq.com
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Permalink | Reply | 13 comments |
headers
Matthias Radestock | 1 Mar 2011 11:07
Favicon

Re: Losing messages, why?

DOBRO Sergei wrote:
> There are six workers. Other clients publish messages to exchange
> "ex.direct" with the routing key which is determined randomly from
> the values of array (routing_key1, routing_key2, ...., routing_key6).
>  So, the workers receive messages randomly too. That's why I can not
> tell you which worker(s) stops receiving messages.

So how do you know that messages are getting lost?

> RabbitMQ and the workers are running on the same machine. Each
> connection's IP is 127.0.0.1. Can you give me some advice? Which
> software to use to access the data flowing across the connection
> RabbitMQ<->worker?

Try wireshark. Or insert our tracer 
http://www.rabbitmq.com/examples.html#tracer.

Matthias.
Permalink | Reply | 12 comments |
headers
Emile Joubert | 1 Mar 2011 11:09
Favicon

Re: socket error while using rabbitmq and a NAT (Network address translation)

Hi,

On 01/03/11 09:14, mysurf mail wrote:
> Hi,
> Thanks for your response.
>
> I followed your advice and here are my results.
>   I see myself in the connection list when I invoke a connection from a
> computer that gets the exception. I see it with "starting"  status and
> instantly it disappears. (I use new exchange name and no exchange is
> created)
>
> I tried to follow your advice about the tracer but i didn't find enough
> info about the file runjava.bat in 2.3.1 (it exists only in the 1.7.0
> package. not the 2.3.1.)
>
> it happens from almost half of the servers. not only the ones with the NAT.
> thanks.

Unfortunately this is not enough information to make any further guesses 
about the cause of the problem. Can you please supply a complete 
(compressed if necessary) broker logfile from one of the brokers where 
this problem occurs.

I've confirmed that runjava.bat exists in the java client and the 
complete windows bundle downloads. You can get the latest version here: 
http://www.rabbitmq.com/java-client.html . It will be helpful if you can 
connect via the tracer and send the output.

Are the failing servers all running the same version of RabbitMQ? Is it 
possible that they have some feature or configuration in common that may 
be the cause of the problem?

Emile
Permalink | Reply | 9 comments |
headers
Simon MacMullen | 1 Mar 2011 13:01
Favicon
Gravatar

Re: Clustering over a WAN

On 28/02/11 21:59, Alexis Richardson wrote:
> On Mon, Feb 28, 2011 at 9:57 PM, Michael Bridgen<mikeb@...>  wrote:
>>> Another approach is to create federation models in new exchange
>>> types.
>>
>> Why have new exchange types -- why not just federate those that exist
>> already?
>
> Yes, that would be spiff too.

FTR the current design for the federation plugin (of which more soon) 
falls between these two - there is a new "x-federation" exchange type, 
but it can emulate any existing type; it doesn't have its own routing logic.

The fact of it being an exchange type may go away in the future, 
although it's actually pretty convenient.

Cheers, Simon

--

-- 
Simon MacMullen
Staff Engineer, RabbitMQ
SpringSource, a division of VMware
Permalink | Reply | 0 comments |

Gmane