Brian Warner | 1 Dec 2006 02:09
Favicon

Re: testFallback2 failing for me

> /usr/local/src/buildbot-0.7.4$ buildbot --version
> Buildbot version: 0.7.4
> Twisted version: 1.3.0rc1
> 
> ERROR: testFallback2 (buildbot.test.test_slaves.Slave)

> Failure: [Failure instance: Traceback: twisted.trial.unittest.FailTest, 2 != 1

> Any ideas?  Is my version of Twisted just too old?

Yeah, basically. There is a race condition that gets triggered by Twisted 1.3
fairly frequently, and some changes went into Twisted 2.0 and later which
generally make it go away. (it involves a __del__ handler which triggers a
lot of activity at a funny time). As far as I can tell it doesn't cause
problems for normal operations, but it does cause unit test failures over
half of the times that I run it, more on some machines than others.

That said, the next release (0.7.5) will probably not support Twisted-1.3.0
(nor python-2.2). I've been using python-2.3 features like sets and "substr"
in "str", and I can clean up some ugly code by not providing backwards
compatibility for some old Twisted-1.3 idioms. I no longer have
metabuildslaves for python-2.2, and the twisted-1.3 builders are showing
errors like the one you saw on a consistent basis.

So 0.7.5 "might work if you're lucky" on either of these platforms, but there
are no promises. Upgrading now would be a splendid idea :).

cheers,
 -Brian

(Continue reading)

Markus Kramer | 3 Dec 2006 16:15
Picon

Re: Win32 Slave Stability Issues?

Ok, i have found some time to take our buildbot down over the weekend 
and debug a little into that problem.
I have traced master slave communication with wireshark, and could see 
that it is actually the master closing down the connection.
After inspecting the logs a little deeper, i found that on start of 
every build, master tries to ping the slave with a 10 Second timeout, 
and if that one fails it declares that slave dead. For some reasons 
windows buildslaves seem to exhibit very odd reply times and do hardly 
manage to answer that ping within 10 seconds when already under load. 
Instead I had to put the timeout up to 5 minutes.  Somehow it seems 
windows buildslaves delay replys on the command channel for quite some 
time (my impression is actually that next reply is only delivered if one 
of the already running build steps on that builder completes, or there 
are no buildsteps in progress).
Is there any method to inspect slaves and see what they are actually 
doing that prevents them from answering pings?

Best regards,

Markus Kramer

Carl Dionne schrieb:
> I'm experiencing a similar problem on windows, but haven't had the time
> to debug it either. 
>
> -----Original Message-----
> From: buildbot-devel-bounces@...
> [mailto:buildbot-devel-bounces@...] On Behalf Of
> Markus Kramer
> Sent: November 29, 2006 8:59 AM
(Continue reading)

Benoit Sigoure | 4 Dec 2006 19:06
Picon
Picon
Picon

Sharing common configurations for different buildfarms

Hello people

I have quite a lot of buildfarms running with buildbot and the configuration is
very redundant. I'd like to share it (all the buildsteps) among different
buildfarms, how would you advice me to do this? I'm not very good in Python so
if you could provide me with an example it'd be great. Can I simply put a
Python file somewhere that defines a class that creates the buildsteps and
import it from the master.cfg's? Where would you advise me to put this Python
file?

Now say that I have a complete BuildFactory (with all the steps) how can I
(deep)copy it and change only a given step (say, the configure step) ?

Thanks
Cheers,

--

-- 
SIGOURE Benoit aka Tsuna
   _____
  /EPITA\ Promo 2008, LRDE

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
Stefan Seefeld | 4 Dec 2006 19:17
Picon
Favicon

Re: Sharing common configurations for different buildfarms

Benoit Sigoure wrote:
> Hello people
> 
> I have quite a lot of buildfarms running with buildbot and the configuration is
> very redundant. I'd like to share it (all the buildsteps) among different
> buildfarms, how would you advice me to do this? I'm not very good in Python so
> if you could provide me with an example it'd be great. Can I simply put a
> Python file somewhere that defines a class that creates the buildsteps and
> import it from the master.cfg's? Where would you advise me to put this Python
> file?

You may put such a python module (or even package) at any place convenient for
you. If this place isn't searched by default you could add the path to sys.path
before importing the module into master.cfg.

> Now say that I have a complete BuildFactory (with all the steps) how can I
> (deep)copy it and change only a given step (say, the configure step) ?

Python has a 'copy' module that offers deep-copying of (almost) arbitrary objects.
However, I think it would be cleaner to somehow parametrize the factory object / function
you describe so you pass in the aruments that vary among the builders.

HTH,
		Stefan

--

-- 

      ...ich hab' noch einen Koffer in Berlin...

-------------------------------------------------------------------------
(Continue reading)

prashanth prakash | 6 Dec 2006 10:07
Picon

help rquired on buildbot.

Hello All,
 
I require some help on configuring the buildbot for automating build process.
 
We are using SVN repository.
 
Right now I have some problem w.r.t the syntax followed in master.cfg file.
 
Please let me know what modifications needs to done in master.cfg file.
Please help me in this regards.
Thanks in advance.
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Buildbot-devel mailing list
Buildbot-devel@...
https://lists.sourceforge.net/lists/listinfo/buildbot-devel
Mark Pauley | 6 Dec 2006 20:01
Picon
Favicon

Re: help rquired on buildbot.

I would suggest you start by reading and fully absorbing the following:

http://www.catb.org/%7Eesr/faqs/smart-questions.html

As for the syntax, it's Python.  You might want to check google for a  
good tutorial, and then read the BuildBot Manual.

cheers,

_Mark

On Dec 6, 2006, at 1:07 AM, prashanth prakash wrote:

> Hello All,
>
> I require some help on configuring the buildbot for automating build  
> process.
>
> We are using SVN repository.
>
> Right now I have some problem w.r.t the syntax followed in  
> master.cfg file.
>
> Please let me know what modifications needs to done in master.cfg  
> file.
> Please help me in this regards.
> Thanks in advance.
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to  
> share your
> opinions on IT & business topics through brief surveys - and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV_______________________________________________
> Buildbot-devel mailing list
> Buildbot-devel@...
> https://lists.sourceforge.net/lists/listinfo/buildbot-devel

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
Grig Gheorghiu | 6 Dec 2006 20:11

Re: help rquired on buildbot.

Your question is extremely vague, with no specifics, so there's no way
we can help you at this point. Here are a couple of howtos I wrote
that might help you get started:

http://agiletesting.blogspot.com/2006/02/continuous-integration-with-buildbot.html
http://agiletesting.blogspot.com/2006/03/running-buildbot-on-various-platforms.html

And here's another good overview:

http://agile.idyll.org/wiki/BuildbotConfiguration

Grig

On 12/6/06, prashanth prakash <prashanthkit@...> wrote:
> Hello All,
>
> I require some help on configuring the buildbot for automating build
> process.
>
> We are using SVN repository.
>
> Right now I have some problem w.r.t the syntax followed in master.cfg file.
>
> Please let me know what modifications needs to done in master.cfg file.
> Please help me in this regards.
> Thanks in advance.
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys - and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
>
> _______________________________________________
> Buildbot-devel mailing list
> Buildbot-devel@...
> https://lists.sourceforge.net/lists/listinfo/buildbot-devel
>
>
>

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
Niklaus Giger | 6 Dec 2006 20:27
Picon
Gravatar

Re: update on the next release (and a suggestion)

Brian Warner wrote:

> Just a status report on how the next release is looking.. 
<..>
> I'm hoping to get this code done over the weekend, and then we can let it
> burn-in for a week or two before cutting the next release.
Using darcs to follow your progress, I noted that you did quite a few work
for more than a weekend.

I am very interested in a few of the new features (e.g. SVNPoller,
FileUp/Download and have tested them in my test environment. Also
the trial tests pass without any errors on my GNU/Debian machine.

Therefore I would like to ask if you could give us a hint when the next
version will be out.

Another question is, that I would have found it nice to specify the
attributes for a downloaded file, as I would like to download some test
scripts which have to be executed on the slave. And I would prefer to
specify in a Shell step "./my_bash.sh", instead of
[ "/bin/bash", "my_bash.sh"]. Would you accept patches for this? Or do you
consider it a too big security risk?

Best regards

Niklaus Giger

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
Robert Helmer | 6 Dec 2006 23:37
Gravatar

more flexible Change class?

Hey,

I am working on making a new type of change object (based on bhearsum's 
bonsaipoller.py) which parses this page:
http://tinderbox.mozilla.org/showbuilds.cgi?tree=Firefox&quickparse=1

I am using it to kick off a series of steps which download the latest 
build and test it.

It works fine for my needs right now, but I am noticing that Change is 
very heavily biased towards revision control changes and not "change" in 
the more generic sense (which the name of the object might lead you to 
believe).

For example, instead of returning who, files, comments, and when it'd 
make more sense for my object to return hostname, status and when. These 
map OK I guess, but would it be desirable to have the change object be 
more generic?

Instead of:

c = changes.Change(who = buildNode['hostname'], files = '', comments = 
buildNode['status'], when = int(buildNode['date']))

Maybe Change could take a dict like:

{'hostname': buildNode['hostname'], 'status':buildNode['status']}

And the "/changes" page would just lay it out like:

--
hostname:
  example-hostname

status:
  success
--

Maybe "who" and "when" would be required, and "files" supported. It'd be 
nice if the html page would lay out a random dict or list like :

{'data': ['1','2','3']}

would be:

data:
 * 1
 * 2
 * 3

I think it might make sense to refactor the common parts of the VCS 
change objects to a subclass of Change, since it seems like (at a 
glance) that they have a lot in common, and it might make supporting new 
systems even easier.

Anyway, not sure if anyone else is interesting in using Buildbot to take 
changes from things that aren't version control systems, but if so 
this'd help me :) Brian, I can put together a patch if this is something 
you'd be interested in. I think it'd be a way of solving the CVS 
revision number silliness we did in the bonsaipoller.py with sticking 
the revision in with the filename.

Thanks,
Rob

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
Brian Warner | 7 Dec 2006 08:58
Favicon

Re: update on the next release (and a suggestion)

> I am very interested in a few of the new features (e.g. SVNPoller,
> FileUp/Download and have tested them in my test environment. Also
> the trial tests pass without any errors on my GNU/Debian machine.
> 
> Therefore I would like to ask if you could give us a hint when the next
> version will be out.

Very soon now. The reconfiguring-builders fix was the last big issue, which I
finally got done about a week ago. The only thing left on my list is to clean
up file upload/download a little bit (specifically adding some helpful error
messages if the slave is too old or if the target file cannot be found). My
current goal is to cut the release this coming weekend.

> Another question is, that I would have found it nice to specify the
> attributes for a downloaded file, as I would like to download some test
> scripts which have to be executed on the slave.

So something like FileDownload(source, dest, mode=0755) ? Sure, I'd accept a
patch for that, after 0.7.5 is out. Don't write it just yet, as I'll probably
be changing FileDownload a little bit before the release, but as soon as
0.7.5 is cut feel free to submit one.

> And I would prefer to specify in a Shell step "./my_bash.sh", instead of [
> "/bin/bash", "my_bash.sh"].

If my_bash.sh has the executable bit set, then command=["./my_bash.sh"] ought
to work (assuming it has a #!/bin/bash line, of course).

> Would you accept patches for this? Or do you consider it a too big security
> risk?

I guess I should settle on how ShellCommands should be specified. I've
certainly had cases myself where I want to use shell metacharacters, things
like "scp ../*.deb host:~/public_html/" and similar. My hunch is that things
like that belong in a Makefile rather than in the Buildbot, and making it
difficult or annoying to use the full power of bash within ShellCommands is a
sort of passive/aggressive way to accomplish that :).

There's not necessarily a security risk in string-based commands. The risk
comes when you are interpolating other strings into them, specifically
strings that come from a hostile party, where you get injection attacks and
games with passing shell metacharacters in through a string that was
"supposed" to be something simple. If you can only use argv array-based
ShellCommands, these threats are a lot less severe. But there aren't a whole
lot of places where we interpolate things into a string-based command
anyways, unless you're using WithProperties a lot.

Would it make sense for this test script that you want to download and
execute on the slave instead be checked in to your source tree? I'd like to
understand the use case a bit more before coming to a decision about whether
to encourage or discourage string-based commands.

[note: by "string-base commands" I mean something like
ShellCommand(command="ls *.foo >stuff.txt") . When you pass a non-array to
command=, the buildslave pretends you had written command=["/bin/sh", "-c",
"ls *.foo >stuff.txt"] instead.]

cheers,
 -Brian

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Gmane