headers
Nick Coghlan | 20 May 2013 09:37
Picon
Favicon

Beaker road map updated

The Bugzilla downtime today ended up providing me with a good 
opportunity to bring the technical road map up to date.

The major changes are to set our objectives for Beaker 1.0 and to 
clearly document beah's current status as fully supported, but not 
actively developed.

* http://beaker-project.org/dev/tech-roadmap.html#objectives-for-beaker-1-0
* http://beaker-project.org/dev/tech-roadmap.html#the-status-of-beah

I also added several references to other projects (like Rickshaw, 
Ansible and cloud-init) that may be of interest for various parts of the 
road map.

The site update also covered the documentation and draft release notes 
for 0.13 changes that have already been merged:

* http://beaker-project.org/docs-develop/whats-new/#unreleased-changes

The full diff is at 
http://git.beaker-project.org/cgit/beaker-project.org/commit/?id=346da204962194b1a95f5aa44954c6589d097990

Cheers,
Nick.

--

-- 
Nick Coghlan
Red Hat Infrastructure Engineering & Development, Brisbane

Test Automation Team Lead
(Continue reading)

Permalink | Reply | 1 comment |
headers
Nick Coghlan | 13 May 2013 09:43
Picon
Favicon

The march towards Beaker 1.0

Beaker has been doing 0.x releases for a long time. We've been talking 
about the release later this month being 1.0, but before we declare 
we've reached that milestone, I'd really like us to be at the point 
where someone could reasonably expect to grab the Beaker RPMs from 
beaker-project.org, and using just our documentation, set up and start 
running their own Beaker instance, including writing new tests using 
either beah or autotest.

We're not there yet, and we're not going to get there this month, so the 
next release will be 0.13 and we'll continue with regular 0.x releases 
until we're happy we have something we consider worthy of the "Beaker 
1.0" title. I don't think we're all that far away - we may need a 0.14, 
but I'll be surprised if we see 0.15.

For planning purposes, I'd like to keep marking the backlog as "1.0" - 
the 0.x releases will then just be snapshots that make useful chunks of 
functionality available to users, rather than having to wait until we're 
done with everything we want to include for the 1.0 milestone.

This will throw out the currently predicted releases in 
http://beaker-project.org/dev/proposals/handling-large-installations.html and 
the associated design proposals, but I'm not really worried about that. 
In some respects, it works out better, as it means we can add the 
"Effective Job Priorities" proposal to our targets for Beaker 1.0, and 
that's the *real* objective of several of the changes we're currently 
working on.

Once we're happy we know what we would like to include in 1.0, then I 
can go back and adjust those design proposals and the tech roadmap 
accordingly.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Nick Coghlan | 13 May 2013 05:44
Picon
Favicon

Defining a more useful component list in Bugzilla

The current component list in Bugzilla has a few limitations, with some 
bugs being thrown at a component more or less at random, since they 
don't fit any of the existing components. In other cases, the component 
is too broad, failing to make appropriate distinctions in the assignments.

Here's the current component list:

     command line
     Doc
     inventory
     qpid
     lab controller
     scheduler
     reports
     tests
     test harness
     web UI

Those actually aren't *that* bad, there are just some missing ones and a 
couple could do with a name change.

Suggested name/scope changes:

     qpid -> notifications (covering both AMQP sending and receiving and 
email sending)
     tests -> test suite
     inventory -> inventory task
     test harness -> beah test harness
     command line -> bkr CLI
(Continue reading)

Permalink | Reply | 4 comments |
headers
shwetha | 7 May 2013 07:48
Picon
Favicon

Handling exit status of the commands execution from all the nodes

Hi All ,

In a multi-node testing environment we have to fail a test case when certain commands on any one of the node fails . For example : if mounting fails on client node , that test case execution should not be continued. A message to be sent to all other machines i.e masternode/node/peer in this case to stop the execution.

How are we going to handle this situation. This is the basic requirement for multi-node testing. We want to check the exit status after every step. For example in most of our testcases the set-up includes :

1) peer probing
2) volume_create
3) volume_info
4) volume_start
5) volume_status
6) mount

In every step we should check the exit status from all the nodes and if any one node has non-zero exit status we should stop that test case and move on to next test case in that task.

-Shwetha
_______________________________________________
Beaker-devel mailing list
Beaker-devel <at> lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/beaker-devel
Permalink | Reply | 0 comments |
headers
Nick Coghlan | 3 May 2013 08:56
Picon
Favicon

Technical roadmap updated

I've just published an update [1] to the tech roadmap at 
http://beaker-project.org/tech-roadmap.html

Three changes of note:

- I moved the "Live Dashboard" down into the exploration section. It 
would still be nice to have it, but it turns out that Graphite's 
rendering API plus the ReloadEvery extension for Firefox ([2]) isn't a 
bad interim solution :)

- I moved the OpenStack integration up into the planned section. oVirt 
Engine is great if you're wanting to run a long lived high availability 
core service, but not so good if you're after throwaway VMs to run a 
single recipe. By contrast, OpenStack's reputation is pretty much the 
exact opposite (great for spinning up instances quickly, less great if 
you then want to keep them running forever), so it will hopefully be a 
better fit for our use case.

- I added Jenkins integration to the "exploration" section. While you 
*can* use Beaker for continuous integration (as Dan's patchbot shows), 
Jenkins is really the king of that space. I see a lot of potential value 
in an approach where Jenkins buildbots are used for initial smoke tests 
on core platforms, and then you can configure a Jenkins plugin to kick 
off a more comprehensive set of integration tests in Beaker.

Cheers,
Nick.

[1] 
http://git.beaker-project.org/cgit/beaker-project.org/commit/?id=43ad402a8f9219f67b45ca1002277ba1bb4511bd
[2] https://addons.mozilla.org/en-US/firefox/addon/reloadevery/

--

-- 
Nick Coghlan
Red Hat Infrastructure Engineering & Development, Brisbane

Test Automation Team Lead
Beaker Development Lead (http://beaker-project.org/)
PulpDist Development Lead (http://pulpdist.readthedocs.org)
_______________________________________________
Beaker-devel mailing list
Beaker-devel <at> lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/beaker-devel
Permalink | Reply | 0 comments |
headers
Nick Coghlan | 3 May 2013 07:48
Picon
Favicon

Patch merge criteria documented

The criteria for marking patches as verified and for passing code review 
are now documented at 
http://beaker-project.org/dev-guide.html#reviewing-a-patch

We'd generally been applying these anyway, but now they're written down 
so people don't need to guess about what we're looking for (and they're 
a useful checklist for us as well).

===============
The "+1 Verified" marker indicates one of the following:

     If it’s a bug fix that is reproduceable and testable, the new test 
case has been verified to fail before the fix, and pass after the fix.
     If it’s a bug fix that is not amenable to an automated test, the 
patch has been verified to fix the bug through some other means (such as 
trying it out manually).
     If it’s a new feature, the feature has been verified to work as 
described.
     If it’s a code change, the test suite has been verified to pass in 
full.
     If it’s a docs change, the docs have been verified to build 
correctly and look right.
     On some rare occasions (for example, fixing a typo in a comment or 
README), it may simply indicate that the patch has been determined not 
to run a risk of breaking the application or documentation.

The "+2 Approved" code review marker should only be granted when all the 
following criteria are met:

     All significant review comments have been addressed, with the aim 
of ensuring the Beaker code remains maintainable rather than 
degenerating over time.
     Whenever practical, automated tests have been added to ensure the 
bug fix or new feature works as expected.
     The code is commented appropriately (for example, explanations or 
issue tracker references are included for any obscure workarounds).
     The documentation (including docstrings) has been updated appropriately
     A release note has been added as described in the What’s New source 
for new features, bug fixes that may break existing workarounds, and any 
changes that require manual steps from system administrators when 
upgrading an existing installation.
     The commit message is correctly formatted with a short summary line 
and any additional continuation lines separated from the summary by a 
blank line.
     For changes driven by a Bugzilla entry, the correct "Bug: NNNNNN" 
reference is present in the commit message (as described above in 
"Submitting your patch").
===============

Cheers,
Nick.

--

-- 
Nick Coghlan
Red Hat Infrastructure Engineering & Development, Brisbane

Test Automation Team Lead
Beaker Development Lead (http://beaker-project.org/)
PulpDist Development Lead (http://pulpdist.readthedocs.org)
_______________________________________________
Beaker-devel mailing list
Beaker-devel <at> lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/beaker-devel
Permalink | Reply | 0 comments |
headers
Nick Coghlan | 30 Apr 2013 11:00
Picon
Favicon

Revised Enhanced User Group proposal on Gerrit

Rather than updating the design proposal directly on the site, I decided 
I wanted some feedback on the current draft first:

http://gerrit.beaker-project.org/#/c/1908/1/dev/proposals/enhanced-user-groups.rst

Key changes:

1. The "job modification" flag on group members is gone, replaced with 
the separate concept of a "submission delegate", who can submit jobs on 
behalf of the group, but cannot modify other group jobs
2. Unlike the previous design, submission delegates retain the ability 
to modify the jobs they submit. The design we discussed in the other 
thread (where you could change the actual submitting *user*) has been 
listed as a deferred feature (because it would require quite a bit more 
additional UI design work)

Another couple of points came up while I was writing it:

- should we allow *completed* single-user jobs to be converted to group 
jobs? We originally didn't allow this due to the problem with getting 
SSH keys installed, but that doesn't apply if the job is already 
completed, and it may be necessary if we want to eventually remove the 
existing not-quite-group-jobs behaviour (I don't think we can remove 
that in 1.0 - we need to allow at least a release for people to migrate 
to the new tools before we take the old mechanism away).
- LDAP derived groups can't be self-administered in the current design, 
but should we at least allow Beaker administrators to nominate 
submission delegates for them?

Cheers,
Nick.

--

-- 
Nick Coghlan
Red Hat Infrastructure Engineering & Development, Brisbane

Python Applications Team Lead
Beaker Development Lead (http://beaker-project.org/)
PulpDist Development Lead (http://pulpdist.readthedocs.org)
_______________________________________________
Beaker-devel mailing list
Beaker-devel <at> lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/beaker-devel
Permalink | Reply | 3 comments |
headers
Nick Coghlan | 30 Apr 2013 02:25
Picon
Favicon

Clarification request for Enhanced User Groups

<moving to the public dev list, no reason for this to be in private email>

On 04/29/2013 08:01 PM, Raymond Mancy wrote:
> Just a couple of questions regarding this following sentence:
>
>    All members of the group with job modification permissions will be able to ack/nack, change priority,
>    edit whiteboard, and change retention tag.
>
> 'All members of the group with job modification permissions', does this actually translate to 'All members
> of the job's group owner'. I'm assuming it is, I was just momentarily confused by it.

No, it's not enough to be a member of the owning group, your membership 
in that group also has to be flagged as "can modify jobs".

See the first part of the proposal (before the "Group ownership" section):

==========
All members of a group are able to submit jobs on behalf of that group.

Group members may also be granted the following additional permissions 
on a group:

     * group ownership
     * group job modification
==========

This means there are two independent permissions on groups, above and 
beyond the "can submit jobs on behalf of the group" capability:

1. Group ownership
   - can add/remove members
   - can change owner status of members
   - can change job modification permissions of members

2. Job modification
   - can modify jobs owned by the group after they are submitted

The intent of this separation is to allow an automated process to be 
given the ability to submit jobs on behalf of the group, *without* 
granting the automated tool's account the ability to modify those jobs 
after creation.

While technically a group owner may not necessarily have job 
modification permissions, they can grant themselves those permissions at 
any time, so it's not a real restriction (from the code's point of view, 
it means we can ignore the group ownership flag when checking for job 
modification permissions).

The default settings when adding a new member to a group are:
   - Group Owner?: False
   - Can Modify Jobs?: True

> Also, is there a reason why we can't afford the group owner the exact same permissions as the
> job submitter (specifically Delete, which is not mentioned)? This would probably make the docs a bit
> more maintainable (as in '...with job modification permissions equal to that of the job submitter'
rather than
> having to update it every time a new job action is created). This would also make the code simpler.

It's meant to be full access - if I missed any current capabilities, 
it's just a mistake in the design.

However, note that part of the redesign is that the job submitter may 
*not* have permissions to make changes after submission, if they're 
submitting the job on behalf of a group and they don't have the job 
modification flag for that group.

Cheers,
Nick.

--

-- 
Nick Coghlan
Red Hat Infrastructure Engineering & Development, Brisbane

Python Applications Team Lead
Beaker Development Lead (http://beaker-project.org/)
PulpDist Development Lead (http://pulpdist.readthedocs.org)
_______________________________________________
Beaker-devel mailing list
Beaker-devel <at> lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/beaker-devel
Permalink | Reply | 7 comments |
headers
Nick Coghlan | 24 Apr 2013 03:53
Picon
Favicon

Beaker 0.12 released

Hello everyone,

On behalf of the Beaker developers, I am pleased to announce that Beaker 
0.12 has been released. More than 50 bugs (including new features) were 
fixed in this release.

Some of the new features introduced in this release are:

- Provisional support for alternative harnesses.
- Live role environment variables.
- New system loan interface and comment fields.
- Disk information in inventory.

A number of other notable changes (some of them potentially backwards 
incompatible) were made. These include:

- Asynchronous updates on job status (to eliminate several race conditions).
- Task names limited to 255 characters (to allow indexing and hence 
faster searching).
- Kickstart sections are properly terminated for RHEL 6 (to ensure 
correct behaviour when making additional repos available during 
installation).

Pre-built packages and source archives are available for download from 
here [1].
To read the release notes in full please go to [2]. The project 
documentation is available at [3].

Due to some regressions [5] found in the initial 0.12 release, the 
current release is actually 0.12.1

If you have any feedback about this release or just want some help, the 
best way to interact with Beaker developers and users is
the #beaker IRC channel on irc.freenode.net. Alternatively, you can post 
your question to the beaker-devel mailing list [4].

[1] http://beaker-project.org/download.html
[2] http://beaker-project.org/docs/whats-new/release-0.12.html
[3] http://beaker-project.org/help.html
[4] https://fedorahosted.org/mailman/listinfo/beaker-devel
[5] http://beaker-project.org/releases/beaker-0.12.1-ChangeLog

--

-- 
Nick Coghlan
Red Hat Infrastructure Engineering & Development, Brisbane

Python Applications Team Lead
Beaker Development Lead (http://beaker-project.org/)
PulpDist Development Lead (http://pulpdist.readthedocs.org)
_______________________________________________
Beaker-devel mailing list
Beaker-devel <at> lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/beaker-devel
Permalink | Reply | 0 comments |
headers
Don Zickus | 28 Mar 2013 17:02
Picon
Favicon

New Beaker API quirks

Hi Dan,

I have been enjoying the ease to code against the new Beaker API.  The
GET, POST, PUSH stuff keeps things really simple.  But during my testing I
noticed a few quirks that Bill P. suggested I email you about for your
opinion.

- When I POST/PUT to a url, if the url has an extra '/' in the path, like

  /recipes/(recipe_id)/tasks/(task_id)//status

the command gets rejected.  Not sure if that is an http thing or a server
thing.  Not to much of a problem, just forces me to clean up my code
(which is good!).  Just thought I would mention it.

- when I PUT a zero length file to the server, it fails with a 400 code.
  Now I should be filtering those (and will now).  But it did take me an
hour before I realized that of my list of files, the rejected ones were of
zero length.  Again could be on purpose, not sure.

- Because the server does not have a mechanism to 'push' and abort the
  client (other than power off), the client could still be running and
trying to pushing results back to the server.

  a POST of a task_result gave me an 'Internal Server Error 500'.  Will
that be the expected response?  I can trap for that and abort the client.

  PUT of logs is still allowed even though the task aborted on the server
side.  Bill thought that should be fixed.  Though I am not sure what the
expectations are.

That's it so far.

Thanks,
Don
_______________________________________________
Beaker-devel mailing list
Beaker-devel <at> lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/beaker-devel
Permalink | Reply | 3 comments |
headers
Amit Saha | 22 Mar 2013 02:00
Picon
Favicon

Notify Job owner before triggering test system scraping

While trying to find the real reason for BZ: 839601 [1] (and other such bugs where reproducing the bug is
non-trivial), Marian Csontos and myself realized that getting access to the test system before it was EWD-ed
would have been quite useful. Considering that Beaker users are mostly proficient in Linux system-admin style
tasks, it would give them the opportunity to try certain basic things such as network connectivity,
checking 
/var/log/messages (and others) to understand and perhaps even single out the reason for being EWD-ed-
whether it was
a call to extend the watchdog failing, the network went down, or whatever may be the reason.

Marian has filed the RFE [2] to this end. I think this may be good to have. We should also have a maximum allowed
timeout value to prevent abuse, of course and a list of things to try, log files to save, etc, etc.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=839601
[2] https://bugzilla.redhat.com/show_bug.cgi?id=923587

Thoughts/comments?

-Amit.

--

-- 
Amit Saha <http://echorand.me>
Infrastructure Engineering and Development
Red Hat, Inc.
_______________________________________________
Beaker-devel mailing list
Beaker-devel <at> lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/beaker-devel
Permalink | Reply | 0 comments |

Gmane