Charles Lepple | 1 Mar 2010 05:21
Picon

console and grid information loss [was: Re: yellow warning on top row of waterfall?]

On Feb 28, 2010, at 2:20 PM, Dustin J. Mitchell wrote:

> Have you checked some of the better views? Console? Grid/TGrid?

The console and grid views still seem to lose a bit of information in  
the name of simplicity. I have heard the horror stories about how hard  
it is to maintain the waterfall code, but it still seems to be the  
most accurate picture of what the entire Buildbot system is doing  
(outside of twistd.log).

I am trying to track down the root cause (or at least a simple test  
case for a bug report), but it does not look like either the console  
or grid views will pick up the status of a re-submitted build. In my  
case, I had not run a 'chmod -R a+w' on the tree before the SVN update  
step, and so the SVN step failed to remove the old tree. I resubmitted  
the build, and after it had completed successfully, there was no  
indication in the grid view that the old build had been re-run, and  
that the second run was successful.

The same thing happened on another slave after I accidentally shut it  
down in the middle of a build. I turned it back on, resubmitted that  
build, and the console and grid still say the build failed.

If someone were to rewrite the waterfall, what would make it less  
fragile?

------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
(Continue reading)

Daniel THOMPSON | 1 Mar 2010 10:47

Re: contrib/git_buildbot.py reports connection lost handling multiple revisions

On 28/02/10 19:16, Dustin J. Mitchell wrote:
> On Fri, Feb 26, 2010 at 9:48 AM, Daniel THOMPSON <daniel.thompson@...> wrote:
>>> Can you send along the "inverse patch" you applied, just so I know
>>> we're talking about the same thing?
>>
>> Sure. See attached.
> 
> Ah, I see what the problem was -- the replacement for the tail
> recursion was not executing under the same deferred, so addChange went
> on interating after the PB connection was dropped, which obviously
> doesn't work :)
> 
> How's this look:
>   http://github.com/djmitche/buildbot/blob/git_buildbot/contrib/git_buildbot.py

Doesn't work as is but was enough for me to take it the rest of the way.
See attached patch.

Three changes:

1. fixed a trivial typo (addChange -> addChanges)
2. make addChange() properly return the deferred from callRemote()
3. hoiked most of iter() into the try: clause so we don't execute it
   on the final iteration

With these in place my trivial test (two changes) works in the same
manner as the code from December.

I also put together a quick script to generate 400 commits. This test
case shows the December code failing (hang after exception) and the new
(Continue reading)

Dan Radez | 1 Mar 2010 16:14
Picon
Favicon
Gravatar

Loki: a buildbot manager


I had started a project a while ago to help me maintain my buildbots.

Long story short, it got so complicated that I've shifted the direction
of the project and changed the dependencies to make it easier for people
to try it out and hopefully use it.

It's now built on Django. So your dependencies are just builtbot, django
and a database library. The app can create/start/stop/hup masters and
slaves and add statuses and steps to them.

Schedulers, multi-server implementations and further integration with
the web ui are planned for development.

I have a screencast linked on the site that should get you going to see
the basics of what it can do. If folks think it's appropriate I'm happy
to link the screencast to the buildbot screeShots page.

I would enjoy your feedback.

Docs, code, etc: https://fedorahosted.org/loki
Screencast: http://www.youtube.com/watch?v=UcxAF2b6O4Y

Dan
Dustin J. Mitchell | 1 Mar 2010 17:16
Favicon
Gravatar

Re: Loki: a buildbot manager

On Mon, Mar 1, 2010 at 9:14 AM, Dan Radez <dradez@...> wrote:
> I have a screencast linked on the site that should get you going to see
> the basics of what it can do. If folks think it's appropriate I'm happy
> to link the screencast to the buildbot screeShots page.

That would be great -- I think that more folks should know about this
great combination.

Have you had a look at the schedulerdb work in the latest commits?  I
wonder how it will help/hinder Loki's interaction with buildbot.

Dustin

--

-- 
Open Source Storage Engineer
http://www.zmanda.com

------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
Dan Radez | 1 Mar 2010 17:46
Picon
Favicon
Gravatar

Re: Loki: a buildbot manager

On 03/01/2010 11:16 AM, Dustin J. Mitchell wrote:
> On Mon, Mar 1, 2010 at 9:14 AM, Dan Radez <dradez@...> wrote:
>> I have a screencast linked on the site that should get you going to see
>> the basics of what it can do. If folks think it's appropriate I'm happy
>> to link the screencast to the buildbot screeShots page.
> 
> That would be great -- I think that more folks should know about this
> great combination.
> 
> Have you had a look at the schedulerdb work in the latest commits?  I
> wonder how it will help/hinder Loki's interaction with buildbot.
> 
> Dustin
> 

I'll post this on the screencasts page.

I've not had a chance to look at the schedulerdb commits, I'll take a
look and see how it impacts my project.

Dan

------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
Axel Hecht | 2 Mar 2010 07:02

Re: console and grid information loss [was: Re: yellow warning on top row of waterfall?]

2010/3/1 Charles Lepple <clepple-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
If someone were to rewrite the waterfall, what would make it less
fragile?

There are a few things that make waterfalls hard in my experience, having written one. OK, just one, but still.

The first is html tables. As soon as something goes somewhat odd, the complete layout breaks down. In particular if you use rowspan (which you should, IMHO, for the sake of smaller output, and better text display).

The next thing is that you need to rotate the flow by 90 degrees in a way twice. At least that's what I recall, and I think my code has it, too. You start off with a flow of events per builder, and then you need to sort those across builders, and then put them back into builders again.

Handling parallel builds per builder is tricky, and either algorithm to do that (add columns, or serialize them in one) has finicky edge cases.

That's off the top of my head.

Axel
------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Buildbot-devel mailing list
Buildbot-devel@...
https://lists.sourceforge.net/lists/listinfo/buildbot-devel
Gregor Doltar | 2 Mar 2010 12:16

buildslave on windows / "All buildslaves appear to be offline, so it's not possible to force this build to execute at this time"

Hi all,

I've set up the following environment:
- buildmaster running on Linux
- buildslave 1 running on Windows
- buildslave 2 running on Linux
- python 2.6.4, twisted 9.0.0, pywin32-214.win32-py2.6 (same on 
buildslave and on buildmaster)
-- in total I've 3 dedicated workstations

The problem:
- buildserver starts and works
- buildslave on Linux works normally
- for windows buildslave following happens:
--- buildmaster shows (page: http://buildmaster:9000/buildslaves) that 
windows buildslave is connected "Slave is currently connected"
--- waterfall page (http://buildmaster:9000/waterfall) shows that 
windows buildslave is offline
--- http://buildmaster:9000/builders/buildbot-full-windows shows that 
all buildslaves are offline. I then try pinging the builder but nothing 
happens

So it seems that communication does not work both ways. Buildslave is 
able to ping buildmaster but vice versa does not work.
I've attempted also the following:
-- placing buildmaster and buildslave on localhost; same problem
-- using different versions of python etc...

I'm quite convinced that firewall or network is not the issue. 
twistd.log doesn't show anything useful.

Any ideas how I should continue ?  Next step I see is debugging...

Regards,
Gregor

------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
Marcus Lindblom | 2 Mar 2010 12:53
Picon
Gravatar

Re: buildslave on windows / "All buildslaves appear to be offline, so it's not possible to force this build to execute at this time"

On 2010-03-02 12:16, Gregor Doltar wrote:

> The problem:
> - buildserver starts and works
> - buildslave on Linux works normally
> - for windows buildslave following happens:
> --- buildmaster shows (page: http://buildmaster:9000/buildslaves) that
> windows buildslave is connected "Slave is currently connected"

Ok. buildslave is connected!

> --- waterfall page (http://buildmaster:9000/waterfall) shows that
> windows buildslave is offline
> --- http://buildmaster:9000/builders/buildbot-full-windows shows that
> all buildslaves are offline. I then try pinging the builder but nothing
> happens

These refer to the builder, not the slave(s). Check that the windows 
builder has the windows slave name correctly set in your master.cfg.

Cheers,
/Marcus

------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
Alexander O'Donovan-Jones | 2 Mar 2010 12:53
Favicon

Re: buildslave on windows / "All buildslaves appear to be offline, so it's not possible to force this build to execute at this time"

Check the slave logs. I run a slave on a mac, and it sometimes runs into issues where it's attempting to use a
directory that doesn't exist. Instead of making the directory it just bombs out.

-----Original Message-----
From: Gregor Doltar [mailto:gregor.doltar@...] 
Sent: Tuesday, March 02, 2010 11:17 AM
To: buildbot-devel@...
Subject: [Buildbot-devel] buildslave on windows / "All buildslaves appear to be offline, so it's not
possible to force this build to execute at this time"

Hi all,

I've set up the following environment:
- buildmaster running on Linux
- buildslave 1 running on Windows
- buildslave 2 running on Linux
- python 2.6.4, twisted 9.0.0, pywin32-214.win32-py2.6 (same on 
buildslave and on buildmaster)
-- in total I've 3 dedicated workstations

The problem:
- buildserver starts and works
- buildslave on Linux works normally
- for windows buildslave following happens:
--- buildmaster shows (page: http://buildmaster:9000/buildslaves) that 
windows buildslave is connected "Slave is currently connected"
--- waterfall page (http://buildmaster:9000/waterfall) shows that 
windows buildslave is offline
--- http://buildmaster:9000/builders/buildbot-full-windows shows that 
all buildslaves are offline. I then try pinging the builder but nothing 
happens

So it seems that communication does not work both ways. Buildslave is 
able to ping buildmaster but vice versa does not work.
I've attempted also the following:
-- placing buildmaster and buildslave on localhost; same problem
-- using different versions of python etc...

I'm quite convinced that firewall or network is not the issue. 
twistd.log doesn't show anything useful.

Any ideas how I should continue ?  Next step I see is debugging...

Regards,
Gregor

------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Buildbot-devel mailing list
Buildbot-devel@...
https://lists.sourceforge.net/lists/listinfo/buildbot-devel

------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
Ravindra Parihar | 2 Mar 2010 13:23
Picon

Trac : scheduler not getting trigger after commit changes in the p4 repository

Hi All,

I am trying to set up buildbot in linux OS.

I have created the build-master and build-slave and configured the master.cfg according to the project requirement.

While starting the buildbot (using buildbot start . )  I am able to start the buildbot without any error.

But Whenever developer commit any change in p4 repository the build is not getting triggered.

I am not sure whether p4poller is polling correctly or their is some problem with scheduler.

Please check the attach master.cfg file and help me to resolve this problem.

Thanks in advance,
Ravi
 

Attachment (master.doc): application/msword, 16 KiB
------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Buildbot-devel mailing list
Buildbot-devel@...
https://lists.sourceforge.net/lists/listinfo/buildbot-devel

Gmane