This is beanstalk-talk, the mailing list for developer and user discussion for beanstalkd and its client libraries. ()

headers

Ben. | 13 May 2013 16:46

How many queues can beanstalkd handle?

Hi,

Assume the following example:

1. We put the status of three lamps into one queue, where (+1) means lamp 1 is on, (-1) means lamp 1 is off etc.

2. We have 1 worker who takes out the status and tells me if the lamp is on or off.

Queue: (+1)(+2)(-3)(-1)(+3)(-2)

The worker takes the first status (+1) and says "Lamp 1 is on". It takes one status at a time and tells me if the lamp is on or off.

Now I have 10.000 lamps and need to increase the speed of the output, so I need to have more workers. I need to make sure that the order of the statuses is not mixed. When the workers handle 5 jobs at the same time it could happen that the lamp is said to be off and on even its on and off.

Solution:

Each lamp gets its own queue => 10.000 lamps

Do you have other ideas how to handle it?

Can Beanstalkd handle 10.000 queues at the same time? What is the limitation, RAM?

Thanks for your input.

Regards

Ben

--
You received this message because you are subscribed to the Google Groups "beanstalk-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beanstalk-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to beanstalk-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
Visit this group at http://groups.google.com/group/beanstalk-talk?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

headers

Jens Rantil | 29 Apr 2013 06:23

"touch" command

Hi,

I'm reading the protocol spec and have a question about the "touch" command. Namely, when the command is executed for a job, how much more time will the job get to process its job? TTR seconds?

Thanks,

Jens

--
You received this message because you are subscribed to the Google Groups "beanstalk-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beanstalk-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to beanstalk-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
Visit this group at http://groups.google.com/group/beanstalk-talk?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

headers

Keith Rarick | 14 Apr 2013 19:21

[ANN] beanstalkd 1.9

This is beanstalkd version 1.9.

As always, there will be no incompatible protocol changes until
version 2.0. A client written for version 1.9 will work
unmodified with any later 1.x release of beanstalkd.

News
----

- compile with debugging symbols by default
- fix case where a job would not time out properly (#174)
- fix build on hurd (#177)
- document the delayed->ready transition via 'kick' (#176)
- fix crash from Conn use after free and double-free (#159)
- fix high CPU use during heavy system load (#169)
- add a unique id and hostname to stats output (#156)

Full list of changes (includes authorship information):  
<http://github.com/kr/beanstalkd/compare/v1.8...v1.9>

Our Urls
--------

Download the 1.9 tarball directly:  
<https://github.com/kr/beanstalkd/archive/v1.9.tar.gz>

Learn all about beanstalk:  
<http://kr.github.com/beanstalkd/>

Talk about beanstalk development or use at:  
<http://groups.google.com/group/beanstalk-talk>

Bugs
----

Please report any bugs to:  
<http://github.com/kr/beanstalkd/issues>

--

-- 
You received this message because you are subscribed to the Google Groups "beanstalk-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beanstalk-talk+unsubscribe@...
To post to this group, send email to beanstalk-talk@...
Visit this group at http://groups.google.com/group/beanstalk-talk?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

headers

Jake Soward | 12 Apr 2013 19:40

Version 1.9

Is there a timeline on when version 1.9 might be released? There have been a few changes since 1.8 that would be very useful to me (mostly deleting delayed jobs).

Thanks

Jake

--
You received this message because you are subscribed to the Google Groups "beanstalk-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beanstalk-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to beanstalk-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
Visit this group at http://groups.google.com/group/beanstalk-talk?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

headers

piyush.goel | 1 Apr 2013 12:24

Deleting binlog files

Hi,

We have recently started using beanstalkd (version 1.4.3) in production with binlogs enabled. I have a couple of questions regarding the same:-

1) Currently the binlog files are getting rotated after they hit 10MB size. Is it possible to change this?

2) The binlog files are growing quite quickly over time and during peak hours the files are consuming over 10GB of space. For clearing up the binlog files, is there any tool available or a simple file deletion would be enough? Are there any possible issues we might run into if we use this approach?

thanks

piyush

--
You received this message because you are subscribed to the Google Groups "beanstalk-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beanstalk-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to beanstalk-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
Visit this group at http://groups.google.com/group/beanstalk-talk?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

headers

schmichael | 30 Mar 2013 07:05

Stateless put and reserve commands

As far as I can tell the only benefit to storing the using and watching state per socket vs. including the tube(s) in every put and reserve command is if you intend on watching a long list of tubes. Otherwise specifying the tube to put a job into or the tubes to reserve per command seem cheap and much more convenient than executing separate use and watch commands.

To this end I've begun implementing stateless put and reserve commands: put-in-tube, reserve-tubes, and reserve-tubes-with-timeout. So far only put-in-tube is implemented.

https://github.com/schmichael/beanstalkd/tree/stateless-commands

I haven't updated the protocol docs or tests yet as I just quickly hacked this together in a couple hours this evening without ever having looked at beanstalkd's source before.

Any feedback is welcome!

--
You received this message because you are subscribed to the Google Groups "beanstalk-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beanstalk-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to beanstalk-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
Visit this group at http://groups.google.com/group/beanstalk-talk?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

headers

Zheng Liu | 22 Mar 2013 23:12

how to list out all the workers and producers for a tube

Hi,

I noticed that there are 'current-watching' and 'current-using' variables per tube, which I can use to get number of workers and producers. But how can I get the ip/hostnames of each of them? This info will be very helpful for debugging.

thanks,

Zheng

--
You received this message because you are subscribed to the Google Groups "beanstalk-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beanstalk-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to beanstalk-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
Visit this group at http://groups.google.com/group/beanstalk-talk?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

headers

David | 21 Mar 2013 15:01

Get all job ids in specific tube

Hi all,

a short and sweet one: I'm wondering how I would go about getting a list of all ids of jobs with a specific status in a specific tube...

Or in other words: I want to be able to get the ids of all ready jobs of tube "tube-name"...

Thanks in advance for your help!

Kind regards,

David

--
You received this message because you are subscribed to the Google Groups "beanstalk-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beanstalk-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to beanstalk-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
Visit this group at http://groups.google.com/group/beanstalk-talk?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

headers

Eddy | 10 Mar 2013 11:12

Running beanstalkd in init script with persistence option.

I created an init script based on upstart guide here : https://github.com/kr/beanstalkd/blob/master/adm/upstart/beanstalkd.conf

Wanting beanstalkd to persist between reboots, I added "-b <dir>" to the script and create /var/lib/beanstalkd folder with write permission but it doesn't seem to start after reboot.

Here's the init script in full:

description "simple, fast work queue"

start on filesystem

stop on runlevel [!2345]

respawn

respawn limit 5 2

exec su nobody -c 'exec /usr/local/bin/beanstalkd -b /var/lib/beanstalkd'

Doing a ps -C beanstalkd returns no process. I am on Ubuntu 12.04. Where did I do wrong?

--
You received this message because you are subscribed to the Google Groups "beanstalk-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beanstalk-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to beanstalk-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
Visit this group at http://groups.google.com/group/beanstalk-talk?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

headers

Perlover | 12 Feb 2013 12:38

Is version 1.8 stable?

Good day!

I plan to use the beanstalkd in my projects
But before i will start i want to ask anybody about stability of version 1.8
I see many reports about core dumps for 1.8 ( https://github.com/kr/beanstalkd/issues/159 )
And as i see that the bug has not been catched as i understand
I plan to use the beanstalkd in my high-load resources for web (for example > 1 million events to beanstalkd for web log items calculation - i plan to send in tubes a log items)
These core dumps of 1.8 version are rarely or not?
Should i use the 1.8 version and may be will you recommend to use more old version like 1.4.6?

Please help to me decide it :)

Thanks!

--
You received this message because you are subscribed to the Google Groups "beanstalk-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beanstalk-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to beanstalk-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
Visit this group at http://groups.google.com/group/beanstalk-talk?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

headers

Jacob | 12 Feb 2013 08:28

Beanstalkd uses a lot more CPU cycles when the system is under high load

I experience that beanstalkd uses a lot CPU cycles when the system is under high load. This, even when beanstalkd is not in use.

This simple test shows the issue in effect:

* beanstalkd v1.8 on an Amazon EC2 Large instance with Ubuntu Linux 12.04.2 LTS.

* No jobs in beanstalkd

* Saturate the available CPU cycles with a couple of nice'd dummy processes, using:

`nice -n 10 python -c "while True: x=2**10"&`

In this test beanstalkd (according to top) uses between 15% and 20% of the CPU cycles. As soon as the high load is stopped, beanstalkd goes back to nearly ~0% CPU.

This observed behaviour is quite unfortunate. So what is happening, and is it expected behaviour?

--
You received this message because you are subscribed to the Google Groups "beanstalk-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beanstalk-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to beanstalk-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
Visit this group at http://groups.google.com/group/beanstalk-talk?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

3	May 2013
18	April 2013
8	March 2013
6	February 2013
6	January 2013
20	December 2012
30	November 2012
53	October 2012
12	September 2012
15	August 2012
14	July 2012
5	June 2012
14	May 2012
22	April 2012
10	March 2012
43	February 2012
31	January 2012
20	December 2011
16	November 2011
5	October 2011
23	September 2011
30	August 2011
18	July 2011
16	June 2011
8	May 2011
6	April 2011
10	March 2011
27	February 2011
29	January 2011
9	December 2010
59	November 2010
12	October 2010
24	September 2010
56	August 2010
32	July 2010
34	June 2010
26	May 2010
14	April 2010
63	March 2010
25	February 2010
27	January 2010
72	December 2009
113	November 2009
95	October 2009
44	September 2009
59	August 2009
14	July 2009
13	June 2009
24	May 2009
41	April 2009
8	March 2009
27	February 2009
69	January 2009
29	December 2008
14	November 2008

Mon	Tue	Wed	Thu	Fri	Sat	Sun

		1	2	3	4	5
6	7	8	9	10	11	12
13	14	15	16	17	18	19
20	21	22	23	24	25	26
27	28	29	30	31