headers
Chris Cooper | 3 Mar 2008 20:35
Gravatar

Condensed error logs from the leak+unit testing tinderboxes -> analysis needed

We've had tinderboxes setup for a while now (~3 months) that create 
debug builds and then run the full suite of unit tests on them with leak 
and bloat logging enabled. We currently have a CentOS and MacOSX box 
running in this config, with 2 Windows machines waiting on increasing 
logging capacity. These machines (qm-leak-centos5-01 and 
qm-leak-macosx-01) are currently reporting to the MozillaTest tinderbox 
tree:

http://tinderbox.mozilla.org/showbuilds.cgi?tree=MozillaTest

There was a great clamor in the Fall of 2007 to get these machines 
setup, but no one seems to have looked at them since, or at least I 
haven't received any feedback about them. I figure either:

a) no one has had time to look,
b) the data they are providing is no good, or
c) the need has gone away.

On the off-chance that it's option a), I went through a day's worth of 
tinderbox logs from each platform about a week ago and parsed out the 
relevant error message to make it more manageable for someone to glance 
at. This more compact analysis is here:

http://wiki.mozilla.org/User:ChrisCooper:UnittestingLeakAnalysis

If someone with more knowledge about the unit tests can have a quick 
look at the failures, it would be appreciated. I'd love to knock out any 
spurious failures and have the data be more meaningful, and maybe even 
have people start looking at the leak/bloat data from these boxes too.
(Continue reading)

Permalink | Reply | 4 comments |
headers
Boris Zbarsky | 3 Mar 2008 21:04
Picon
Favicon

Re: Condensed error logs from the leak+unit testing tinderboxes -> analysis needed

Chris Cooper wrote:
> http://wiki.mozilla.org/User:ChrisCooper:UnittestingLeakAnalysis
> 
> If someone with more knowledge about the unit tests can have a quick 
> look at the failures, it would be appreciated.

Please get a bug filed on that fastload file thing?  That should get fixed..

-Boris
Permalink | Reply | 1 comment |
headers
L. David Baron | 3 Mar 2008 22:01
Gravatar

Re: Condensed error logs from the leak+unit testing tinderboxes -> analysis needed

On Monday 2008-03-03 14:35 -0500, Chris Cooper wrote:
> a) no one has had time to look,

I did look once, but they weren't actually on tinderbox when I
looked, and I didn't have time to figure out who I should bug about
that.  (I think once they were actually explicitly hidden by the
tinderbox admin UI, and I unhid them.)

I think one of the issues with them is that we're trying to add
multiple new conditions for orange/red all at the same time, and it
might be more practical to add them in steps, as we can get things
green.  That said, it may well not be that much work to get them
green.

It might also be helpful to have separate test tinderbox trees for
different things rather than using MozillaTest for everything so we
can load a page showing just these boxes.

-David

--

-- 
L. David Baron                                 http://dbaron.org/
Mozilla Corporation                       http://www.mozilla.com/
Permalink | Reply | 1 comment |
headers
L. David Baron | 3 Mar 2008 22:21
Gravatar

Re: Condensed error logs from the leak+unit testing tinderboxes -> analysis needed

On Monday 2008-03-03 13:01 -0800, L. David Baron wrote:
> I think one of the issues with them is that we're trying to add
> multiple new conditions for orange/red all at the same time, and it
> might be more practical to add them in steps, as we can get things
> green.  That said, it may well not be that much work to get them
> green.

For example, if most of the failures are because assertions are
fatal (I don't know whether this is the case), perhaps we could have
code that counts the number of assertions in the log and shows it in
the tinderbox display (and maybe turns things orange if it's greater
than a fixed number in the config that we could lower as we fix
them) instead of actually making them turn the tinderbox red for any
assertions.

-David

--

-- 
L. David Baron                                 http://dbaron.org/
Mozilla Corporation                       http://www.mozilla.com/
Permalink | Reply | 0 comments |
headers
Chris Cooper | 3 Mar 2008 23:50
Gravatar

Re: Condensed error logs from the leak+unit testing tinderboxes -> analysis needed

Boris Zbarsky wrote:
> Chris Cooper wrote:
>> http://wiki.mozilla.org/User:ChrisCooper:UnittestingLeakAnalysis
>>
> Please get a bug filed on that fastload file thing?  That should get fixed..

Filed: https://bugzilla.mozilla.org/show_bug.cgi?id=420776

cheers,
--
coop
Permalink | Reply | 0 comments |
headers
Mark Banner | 10 Mar 2008 23:36

Suggestions for running Mochitests on Thunderbird

I am currently making some initial investigations into running 
mochitests on Thunderbird.

I'm not too worried if the core tests don't/won't run (e.g. they will 
eventually be covered by xulrunner automated testing), but we really do 
need mochitests for the MailNews specific items.

The main problem here, of course, is that we haven't got a browser 
window when we start up.

I've tracked down one problem (not starting up at all) to the http url 
on the command line, obviously we'll need that, or a way of handling it.

One thought I've had is to provide a test-only command line handler that 
could handle '-test' or even the http url, which would then do something 
special on starting TB - is it feasable to create a browser window somehow?

Any thoughts would be very much apprechiated.

Standard8
Permalink | Reply | 1 comment |
headers
Mike Shaver | 10 Mar 2008 23:54
Picon

Re: Suggestions for running Mochitests on Thunderbird

On Mon, Mar 10, 2008 at 6:36 PM, Mark Banner
<bugzilla <at> invalid.standard8.plus.com> wrote:
> I am currently making some initial investigations into running
>  mochitests on Thunderbird.

Woo!

>  One thought I've had is to provide a test-only command line handler that
>  could handle '-test' or even the http url, which would then do something
>  special on starting TB - is it feasable to create a browser window somehow?

The --chrome mode for runtests.py might be what you want here, if
Thunderbird honours it.

(Not a performance topic, but trying to move this to dev.quality will
break the thread, I'm sure.)

Mike
Permalink | Reply | 0 comments |
headers
Mark Banner | 11 Mar 2008 21:43

Re: Suggestions for running Mochitests on Thunderbird

Mike Shaver wrote:
> (Not a performance topic, but trying to move this to dev.quality will
> break the thread, I'm sure.)

I've just done a new post in dev.quality, probably not right, but at 
least it should get the right attention.

Standard8
Permalink | Reply | 0 comments |
headers
L. David Baron | 14 Mar 2008 07:52
Gravatar

Re: Condensed error logs from the leak+unit testing tinderboxes -> analysis needed

On Monday 2008-03-03 14:35 -0500, Chris Cooper wrote:
> We've had tinderboxes setup for a while now (~3 months) that create 
> debug builds and then run the full suite of unit tests on them with leak 
> and bloat logging enabled. We currently have a CentOS and MacOSX box 
> running in this config, with 2 Windows machines waiting on increasing 
> logging capacity. These machines (qm-leak-centos5-01 and 
> qm-leak-macosx-01) are currently reporting to the MozillaTest tinderbox 
> tree:
> 
> http://tinderbox.mozilla.org/showbuilds.cgi?tree=MozillaTest

They seem to have disappeared.  Did they move to another tree, or
just stop running for some reason?

-David

--

-- 
L. David Baron                                 http://dbaron.org/
Mozilla Corporation                       http://www.mozilla.com/
Permalink | Reply | 1 comment |
headers
Chris Cooper | 14 Mar 2008 14:50
Gravatar

Re: Condensed error logs from the leak+unit testing tinderboxes -> analysis needed

L. David Baron wrote:
> On Monday 2008-03-03 14:35 -0500, Chris Cooper wrote:
>> http://tinderbox.mozilla.org/showbuilds.cgi?tree=MozillaTest
> 
> They seem to have disappeared.  Did they move to another tree, or
> just stop running for some reason?

Centos was hung at a login manager prompt during mochitest, and the Mac 
slave had simply failed to shutdown the browser after the mochitests. 
I've restarted both slaves.

cheers,
--
coop
Permalink | Reply | 0 comments |

Gmane