Donovan | 1 Jun 2010 09:22
Picon
Gravatar

rabbitmq-jsonrpc-channel on port 80

Hi,

I'm trying to set up rabbit_mochiweb to be served on port 80 so that I
can invoke JSON-RPC methods without any firewall configuration. On
Lucid Lynx I configure /etc/rabbitmq/rabbitmq.conf:

[
    {rabbit_mochiweb, [{port, 80}]}
].

and get:

Error: {node_start_failed,normal}

Crash dump was written to: erl_crash.dump
Kernel pid terminated (application_controller)
({application_start_failure,rabbit_mochiweb,{shutdown,
{rabbit_mochiweb_app,start,[normal,[]]}}})

If I change the port to any value above 1023, it works, which makes me
think this a running as root issue. Any ideas how to make this work?

Cheers,
Donovan.

Marek Majkowski | 1 Jun 2010 13:25
Picon
Gravatar

Re: rabbitmq-jsonrpc-channel on port 80

On Tue, Jun 1, 2010 at 08:22, Donovan <donovanhide <at> gmail.com> wrote:
> I'm trying to set up rabbit_mochiweb to be served on port 80 so that I
> can invoke JSON-RPC methods without any firewall configuration. On
> Lucid Lynx I configure /etc/rabbitmq/rabbitmq.conf:
>
> [
>    {rabbit_mochiweb, [{port, 80}]}
> ].
>
> If I change the port to any value above 1023, it works, which makes me
> think this a running as root issue. Any ideas how to make this work?

Hi,

Rabbitmq is starting as an unprivileged user, so it can't bind to ports
less than 1024. I'd suggest using some intermediate layer for that,
maybe haproxy?

Steps:
apt-get install haproxy

Edit /etc/default/haproxy to enable it.
Modify config, to something like

global
        maxconn 2000
        user haproxy
        group haproxy
        daemon

(Continue reading)

Sylvain Hellegouarch | 1 Jun 2010 13:35
Gravatar

Re: detecting publish failure across restart



On Mon, May 31, 2010 at 10:30 PM, David Wragg <david <at> rabbitmq.com> wrote:
Hi Sylvain,

Sylvain Hellegouarch <sh-5mkEVRNLbqMdnm+yROfE0A@public.gmane.org> writes:
> I'm curious, in the general case, what's the cost of using a transaction?
> Does it consume more resources? Roughly, what takes place within
> RabbitMQ?

There is indeed a cost associated with a transaction.  One source of
overhead is the fact that the AMQP tx.commit method is synchronous, and
so involves a round-trip over the network to the server.  Another is the
fact that when the server commits a transaction, it must make sure that
the relevant effects of a transaction have reached stable storage
(e.g. that a persistent message that reaches a queue has been stored on
disk).  My colleagues might be able to list other costs.

It's difficult to quantify these costs in general, but I think its fair
to say that they have a significant impact compared with
non-transactional uses of AMQP.  The MulticastMain tool included with
our Java client library has command line options that allow the impact
to be measured in various scenarios.

David

Cheers for that feedback. Much appreciated. I will try to perform some measurements and see both the server and client behave.


--
- Sylvain
http://www.defuze.org
http://twitter.com/lawouach
<div>
<br><br><div class="gmail_quote">On Mon, May 31, 2010 at 10:30 PM, David Wragg <span dir="ltr">&lt;<a href="mailto:david@...">david <at> rabbitmq.com</a>&gt;</span> wrote:<br><blockquote class="gmail_quote">
Hi Sylvain,<br><div class="im">
<br>
Sylvain Hellegouarch &lt;<a href="mailto:sh@...">sh@...</a>&gt; writes:<br>
&gt; I'm curious, in the general case, what's the cost of using a transaction?<br>
&gt; Does it consume more resources? Roughly, what takes place within<br>
&gt; RabbitMQ?<br><br>
</div>There is indeed a cost associated with a transaction. &nbsp;One source of<br>
overhead is the fact that the AMQP tx.commit method is synchronous, and<br>
so involves a round-trip over the network to the server. &nbsp;Another is the<br>
fact that when the server commits a transaction, it must make sure that<br>
the relevant effects of a transaction have reached stable storage<br>
(e.g. that a persistent message that reaches a queue has been stored on<br>
disk). &nbsp;My colleagues might be able to list other costs.<br><br>
It's difficult to quantify these costs in general, but I think its fair<br>
to say that they have a significant impact compared with<br>
non-transactional uses of AMQP. &nbsp;The MulticastMain tool included with<br>
our Java client library has command line options that allow the impact<br>
to be measured in various scenarios.<br><div>
<div></div>
<div class="h5">
<br>
David<br>
</div>
</div>
</blockquote>
<div>
<br>Cheers for that feedback. Much appreciated. I will try to perform some measurements and see both the server and client behave.<br><br><br>
</div>
</div>-- <br>- Sylvain<br><a href="http://www.defuze.org">http://www.defuze.org</a><br><a href="http://twitter.com/lawouach">http://twitter.com/lawouach</a><br>
</div>
Marek Majkowski | 1 Jun 2010 14:07
Picon
Gravatar

Re: RabbitMQ does not start with rabbit-status plugin - broker is running

On Mon, May 31, 2010 at 13:28, Steverding, Kai <Kai.Steverding@...> wrote:
> Hi !
>
> I have a Rabbit node running on Ubtuntu 10.04 64 bit (running RabbitMQ
> 1.7.2). When I put the rabbit-status plugin into the plugin dir and do a
> rabbitmq-active-plugins, the rabbitmq does not start anymore.

I tried to repeat that on Ubuntu 10.04 (Lucid Lynx) 64bit, but it
works fine for me.
Here are my steps:

sudo apt-get install rabbitmq-server curl
cd /tmp
wget http://dev.lshift.net/majek/rabbitmq-status_1.7.0.tar.gz
sudo mkdir /usr/lib/rabbitmq/lib/rabbitmq_server-1.7.2/plugins
cd /usr/lib/rabbitmq/lib/rabbitmq_server-1.7.2/plugins
sudo tar xvzf ~/rabbitmq-status_1.7.0.tar.gz
sudo rabbitmq-activate-plugins
sudo /etc/init.d/rabbitmq-server restart

And RabbitMQ happily lisetns to http://guest:guest-savSHZN5Fh8qMp+WYRx65w <at> public.gmane.org:55672/

Cheers,
  Marek Majkowski

Steverding, Kai | 1 Jun 2010 14:21
Favicon

Re: RabbitMQ does not start with rabbit-status plugin - broker is running

Strange !
I did the same on another Ubuntu system without rabbit and get the same error. I followed your instructions
step by step

sudo apt-get install rabbitmq-server
Paketlisten werden gelesen... Fertig
Abhängigkeitsbaum wird aufgebaut       
Status-Informationen einlesen... Fertig
rabbitmq-server ist schon die neueste Version.
Die folgenden Pakete wurden automatisch installiert und werden nicht länger benötigt:
  libmysqlclient15off libapr1 libaprutil1-ldap netcat-openbsd libaprutil1-dbd-sqlite3 ssl-cert libaprutil1
Verwenden Sie »apt-get autoremove«, um sie zu entfernen.
0 aktualisiert, 0 neu installiert, 0 zu entfernen und 11 nicht aktualisiert.
1 nicht vollständig installiert oder entfernt.
Nach dieser Operation werden 0B Plattenplatz zusätzlich benutzt.
Richte rabbitmq-server ein (1.7.2-1ubuntu1) ...
Starting rabbitmq-server: SUCCESS
rabbitmq-server.

root <at> rabbit-cl1:~#
root <at> rabbit-cl1:~# cd /tmp
root <at> rabbit-cl1:/tmp# wget http://dev.lshift.net/majek/rabbitmq-status_1.7.0.tar.gz

--2010-06-01 14:16:11--  http://dev.lshift.net/majek/rabbitmq-status_1.7.0.tar.gz

Verbindungsaufbau zu 10.10.1.1:3128... verbunden.
Proxy Anforderung gesendet, warte auf Antwort... 200 OK
Länge: 259243 (253K) [application/x-gzip]
In »»rabbitmq-status_1.7.0.tar.gz««
speichern.

100%[==========================================================================================>]
259.243     67,6K/s   in 3,7s    

2010-06-01 14:16:17 (67,6 KB/s) - »»rabbitmq-status_1.7.0.tar.gz«« gespeichert [259243/259243]
root <at> rabbit-cl1:/tmp# sudo mkdir /usr/lib/rabbitmq/lib/rabbitmq_server-1.7.2/plugins
root <at> rabbit-cl1:/tmp# cd /usr/lib/rabbitmq/lib/rabbitmq_server-1.7.2/plugins
root <at> rabbit-cl1:/usr/lib/rabbitmq/lib/rabbitmq_server-1.7.2/plugins# sudo tar xvzf /tmp/rabbitmq-status_1.7.0.tar.gz
rabbitmq-mochiweb.ez
mochiweb.ez
rabbit_status.ez
root <at> rabbit-cl1:/usr/lib/rabbitmq/lib/rabbitmq_server-1.7.2/plugins
root <at> rabbit-cl1:/usr/lib/rabbitmq/lib/rabbitmq_server-1.7.2/plugins# sudo rabbitmq-activate-plugins
root <at> rabbit-cl1:/usr/lib/rabbitmq/lib/rabbitmq_server-1.7.2/plugins#
/etc/init.d/rabbitmq-server restart
Restarting rabbitmq-server: FAILED - check /var/log/rabbitmq/startup_log, _err
rabbitmq-server.
root <at> rabbit-cl1:/usr/lib/rabbitmq/lib/rabbitmq_server-1.7.2/plugins#

Mit freundlichen Grüßen
Kai Steverding

_________________________________________________
GWS Gesellschaft für Warenwirtschafts-Systeme mbH
Krögerweg 10 - 48155 Münster
Kai Steverding
Branchensoftware-Entwicklung
Tel:   +49 (251) 7000-02
Fax:   +49 (251) 7000-3999
mailto:Kai.Steverding <at> gws.ms
Internet: http://www.gws.ms



GWS Gesellschaft für Warenwirtschafts-Systeme mbH
Münster, München, Nürnberg, Leonberg
Geschäftsführer: Dipl. Betriebswirt Helmut Benefader
Sitz: Münster (Westf.)
Amtsgericht Münster HRB 3844







-----Ursprüngliche Nachricht-----
Von: Marek Majkowski [mailto:majek04 <at> gmail.com] 
Gesendet: Dienstag, 1. Juni 2010 14:07
An: Steverding, Kai
Cc: rabbitmq-discuss <at> lists.rabbitmq.com
Betreff: Re: [rabbitmq-discuss] RabbitMQ does not start with rabbit-status plugin - broker is running

On Mon, May 31, 2010 at 13:28, Steverding, Kai <Kai.Steverding <at> gws.ms> wrote:
> Hi !
>
> I have a Rabbit node running on Ubtuntu 10.04 64 bit (running RabbitMQ
> 1.7.2). When I put the rabbit-status plugin into the plugin dir and do a
> rabbitmq-active-plugins, the rabbitmq does not start anymore.

I tried to repeat that on Ubuntu 10.04 (Lucid Lynx) 64bit, but it
works fine for me.
Here are my steps:

sudo apt-get install rabbitmq-server curl
cd /tmp
wget http://dev.lshift.net/majek/rabbitmq-status_1.7.0.tar.gz

sudo mkdir /usr/lib/rabbitmq/lib/rabbitmq_server-1.7.2/plugins
cd /usr/lib/rabbitmq/lib/rabbitmq_server-1.7.2/plugins
sudo tar xvzf ~/rabbitmq-status_1.7.0.tar.gz
sudo rabbitmq-activate-plugins
sudo /etc/init.d/rabbitmq-server restart

And RabbitMQ happily lisetns to http://guest:guest <at> 127.0.0.1:55672/


Cheers,
  Marek Majkowski
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss <at> lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Donovan | 1 Jun 2010 15:10
Picon
Gravatar

Re: rabbitmq-jsonrpc-channel on port 80

Hi Marek,

thanks for getting back with that solution. I had already done a
similar proxy with nginx, but haproxy looks like a lighter weight
solution!

If I wanted to elevate the privileges of the rabbitmq user that gets
created with the ubuntu package, would that work? It seems a bit
counter-intuitive to have two web servers installed, when only one is
needed.

Cheers,
Donovan.

On Jun 1, 12:25 pm, Marek Majkowski <maje...@...> wrote:
> On Tue, Jun 1, 2010 at 08:22, Donovan <donovanh...@...> wrote:
> > I'm trying to set up rabbit_mochiweb to be served on port 80 so that I
> > can invoke JSON-RPC methods without any firewall configuration. On
> > Lucid Lynx I configure /etc/rabbitmq/rabbitmq.conf:
>
> > [
> >    {rabbit_mochiweb, [{port, 80}]}
> > ].
>
> > If I change the port to any value above 1023, it works, which makes me
> > think this a running as root issue. Any ideas how to make this work?
>
> Hi,
>
> Rabbitmq is starting as an unprivileged user, so it can't bind to ports
> less than 1024. I'd suggest using some intermediate layer for that,
> maybe haproxy?
>
> Steps:
> apt-get install haproxy
>
> Edit /etc/default/haproxy to enable it.
> Modify config, to something like
>
> global
>         maxconn 2000
>         user haproxy
>         group haproxy
>         daemon
>
> listen  http-server 0.0.0.0:80
>         mode    http
>         balance roundrobin
>         contimeout      5000
>         clitimeout      50000
>         srvtimeout      50000
>         server  inst00 127.0.0.1:8000 check inter 2000 fall 3
>
> Change 8000 to whatever port you need, like 55672.
>
> Cheers,
>   Marek Majkowski
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-disc...@...://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

Matthew Sackman | 1 Jun 2010 15:28
Favicon

Re: rabbitmq-jsonrpc-channel on port 80

On Tue, Jun 01, 2010 at 06:10:19AM -0700, Donovan wrote:
> thanks for getting back with that solution. I had already done a
> similar proxy with nginx, but haproxy looks like a lighter weight
> solution!

Err, no proxy required. iptables should be sufficient.

iptables -t nat -A PREROUTING -d 127.0.0.1 -p tcp --dport 80 -j DNAT --to-destination 127.0.0.1:55672

(or something like that!)

Matthew

Donovan | 1 Jun 2010 16:32
Picon
Gravatar

Re: rabbitmq-jsonrpc-channel on port 80

Hi Matthew,

thanks for the iptables pointer! Had to use the real ip address rather
than 127.0.0.1 to make it work:

sudo iptables -t nat -A PREROUTING -p tcp -d 172.16.200.10 --dport 80 -
j DNAT --to 172.16.200.10:55672

for some reason!

Cheers,
Donovan.

On Jun 1, 2:28 pm, Matthew Sackman <matt...@...> wrote:
> On Tue, Jun 01, 2010 at 06:10:19AM -0700, Donovan wrote:
> > thanks for getting back with that solution. I had already done a
> > similar proxy with nginx, but haproxy looks like a lighter weight
> > solution!
>
> Err, no proxy required. iptables should be sufficient.
>
> iptables -t nat -A PREROUTING -d 127.0.0.1 -p tcp --dport 80 -j DNAT --to-destination 127.0.0.1:55672
>
> (or something like that!)
>
> Matthew
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-disc...@...://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

Matthew Sackman | 1 Jun 2010 16:53
Favicon

Re: Can the consumer know who sent the message?

Hi Bob,

On Fri, May 28, 2010 at 03:26:44PM -0500, Bob Goldstein wrote:
> Is there a way for the message consumer to know the rabbitmq userid of
> the process that sent the message?

No, not by default, and intentially not by design. Of course, there's
nothing to stop you putting any information, including userid, into any
of the headers, property tables, or even the payload, when publishing a
message.

> I realize there is the props hash, which has a user_id attribute.
> But I believe props is filled in by the sender, so the sender
> could forge anything.

Yes, exactly. But non-repudiation is really something that AMQP isn't
trying to solve anyway - if you want that, then you need to layer all
sorts of extra crypto on top anyway.

> The best I can figure is to create a queue, and make sure that a
> particular sender is the only one with privs to write to that queue.
> However, that seems cumbersome, fragile, and not particularly scalable.

Well, see above about doing it "properly", but you could have something
like an rpc service which hands out private/anonymous queues to
publishers and forwards the queue name to the consumer which is then
able to subscribe to the queue. There you'd be relying on the queue name
from being unforgeable, and as you say, you'd probably want to set up
some additional restrictions so that only the publisher in question can
publish. Queues aren't /that/ heavyweight - 10s of thousands are
perfectly feasible, but it depends on quite what your scalability
requirements are.

Matthew

Marek Majkowski | 1 Jun 2010 17:05
Picon
Gravatar

Re: rabbitmq-jsonrpc-channel on port 80

On Tue, Jun 1, 2010 at 14:10, Donovan <donovanhide@...> wrote:
> If I wanted to elevate the privileges of the rabbitmq user that gets
> created with the ubuntu package, would that work? It seems a bit
> counter-intuitive to have two web servers installed, when only one is
> needed.

Don't do that, running Rabbitmq as root is a bad idea.

Marek


Gmane