headers
Ron Jeffries | 1 Jun 2010 01:03
Picon
Favicon
Gravatar

Re: Shouldnt done include everything.

Hello, xtremenilanjan.  On Monday, May 31, 2010, at 9:38:43 AM,
you wrote:

> Some agile teams I have spoken to and a few accounts I have read,
> do a certain amount of testing after the iteration is complete. 
> The idea is that acceptance tests are done, but there are still
> minor defects which need to be closed.  In some cases people do
> exploratory testing, performance testing etc. in the next iteration.

> Shouldn't "done" include everything?  The purpose from what I
> understand is to keep the concept of "complete" simple - done or
> not done and get a customer buy-in.

> I can understand having performance tests outside the iteration. 
> However, I don't see why exploratory testing would not fall into a single iteration.

Clearly it is difficult to do all the exploratory testing within the
iteration, unless programmers stop programming before the end. (They
could just "fix bugs" at the end but in that case I would downgrade
them for having enough bugs to fix.)

However, if exploratory testing finds defects, I would think that
one or both of these things is true:

  1. Acceptance criteria are not clear;
  2. Automated testing is not strong enough.

So if exploratory testing is finding defects, the team has some
learning to do. If it isn't finding defects, it can still be finding
"interesting things" which can be turned into new stories.
(Continue reading)

Permalink | Reply | 47 comments |
headers
Adam Sroka | 1 Jun 2010 02:03
Picon
Gravatar

Re: Shouldnt done include everything.

Bill Wake has an old article on his site that describes the nature of
good stories and tasks. It says that tasks should be "SMART: Specific,
Measurable, Achievable, Relevant, and Time-Boxed."
(http://xp123.com/xplor/xp0308/ also reproduced in Mike Cohn's /User
Stories Applied/.)

It seems to me that exploratory testing has none of those qualities.
Rather, it is more like many other activities that we do all the time:
building, integrating, running the tests for regression, refactoring,
talking to the customer, etc. These are vital to quality and our
continued progress, but not necessarily specific to the story at hand.

Exploratory testing is not meant to just catch bugs in the feature you
were working on. It is meant to find bugs in the cracks between the
stories. Those cracks don't belong to any story. So, exploring them
can't really belong to the definition of done for that story.

We also can't say, "We'll do exploratory testing until we're done,"
because exploratory testing is never done. It's like refactoring. We
don't "refactor until we're done." We refactor until we are
sufficiently confident that we have a clean enough and simple enough
design /for now./ We could always do more refactoring. We could spend
weeks and weeks on it. The same thing is true of exploratory testing.

It is reasonable to say that we will do some exploratory testing every
time we implement a new feature. It is not reasonable to say that we
will constrain the scope of such testing such that we know when we are
done, because that is not exploratory testing.

...
(Continue reading)

Permalink | Reply | 3 comments |
headers
xtremenilanjan | 1 Jun 2010 16:18
Picon
Favicon

Re: Shouldnt done include everything.

This is aside from my original question-  

Isn't it setting the bar high - if we expect programmers to keep programming till the end of the iteration?
:-)  Do let me know if I should expect this.

I am thinking more from the point of view of a software product team

--- In extremeprogramming <at> yahoogroups.com, Ron Jeffries <ronjeffries <at> ...> wrote:
>
> Hello, xtremenilanjan.  On Monday, May 31, 2010, at 9:38:43 AM,
> you wrote:
> 
> > Some agile teams I have spoken to and a few accounts I have read,
> > do a certain amount of testing after the iteration is complete. 
> > The idea is that acceptance tests are done, but there are still
> > minor defects which need to be closed.  In some cases people do
> > exploratory testing, performance testing etc. in the next iteration.
> 
> > Shouldn't "done" include everything?  The purpose from what I
> > understand is to keep the concept of "complete" simple - done or
> > not done and get a customer buy-in.
> 
> > I can understand having performance tests outside the iteration. 
> > However, I don't see why exploratory testing would not fall into a single iteration.
> 
> Clearly it is difficult to do all the exploratory testing within the
> iteration, unless programmers stop programming before the end. (They
> could just "fix bugs" at the end but in that case I would downgrade
> them for having enough bugs to fix.)
>
(Continue reading)

Permalink | Reply | 42 comments |
headers
xtremenilanjan | 1 Jun 2010 16:40
Picon
Favicon

Re: Shouldnt done include everything.

This is a great response.  however, I think my mention of exploratory testing has opened up issues I wasn't
thinking about.  I also need to read up Mike Cohn, Ron Jefferies.  In the meantime,...

My context: Developing software products/shrink wrapped

Despite all XP practices, I would expect a good tester/exploratory tester will find significant defects. 
In XP, he/she would endeavor to find these defects as soon as possible, maybe by clarifying user stories or
improving developer tests.  However, I still expect that the tester would find significant defects
during/latter part of the iteration.  I think that would require creative/analytical/thinking skills
that XP practices can't guarantee.  

Given that, I would plan some exploratory testing for the feature in the iteration.  I would plan some time
and have some mechanism to determine if that time is sufficient.

It's possible that I don't really understand XP (as of now).

--- In extremeprogramming <at> yahoogroups.com, Adam Sroka <adam.sroka <at> ...> wrote:
>
> Bill Wake has an old article on his site that describes the nature of
> good stories and tasks. It says that tasks should be "SMART: Specific,
> Measurable, Achievable, Relevant, and Time-Boxed."
> (http://xp123.com/xplor/xp0308/ also reproduced in Mike Cohn's /User
> Stories Applied/.)
> 
> It seems to me that exploratory testing has none of those qualities.
> Rather, it is more like many other activities that we do all the time:
> building, integrating, running the tests for regression, refactoring,
> talking to the customer, etc. These are vital to quality and our
> continued progress, but not necessarily specific to the story at hand.
>
(Continue reading)

Permalink | Reply | 2 comments |
headers
Curtis Cooley | 1 Jun 2010 16:42
Picon

Re: Re: Shouldnt done include everything.

On Tue, Jun 1, 2010 at 7:18 AM, xtremenilanjan <xtremenilanjan <at> yahoo.com>wrote:

>
>
> This is aside from my original question-
>
> Isn't it setting the bar high - if we expect programmers to keep
> programming till the end of the iteration? :-) Do let me know if I should
> expect this.
>
> What else would the do?

XP taught me about fear. It taught me how much it controls software
development, how to recognize it, and how to deal with it. I feel your
question arises from the fear that the changes the developers make to the
system won't work. XP has practices in place to help ensure that the changes
do work. And that they do not break other parts of the system. XP done well
allows the programmers to program right up to the demo. XP done well frees
you from fear's control over your project.

--

-- 
Curtis Cooley
curtis.cooley <at> gmail.com
home:http://curtiscooley.com
blog:http://ponderingobjectorienteddesign.blogspot.com
===============
Leadership is a potent combination of strategy and character. But if you
must be without one, be without the strategy.
-- H. Norman Schwarzkopf
(Continue reading)

Permalink | Reply | 0 comments |
headers
Steven Gordon | 1 Jun 2010 17:38
Picon

Re: Re: Shouldnt done include everything.

On Tue, Jun 1, 2010 at 7:40 AM, xtremenilanjan <xtremenilanjan <at> yahoo.com>wrote:

>
>
> This is a great response. however, I think my mention of exploratory
> testing has opened up issues I wasn't thinking about. I also need to read up
> Mike Cohn, Ron Jefferies. In the meantime,...
>
> My context: Developing software products/shrink wrapped
>
> Despite all XP practices, I would expect a good tester/exploratory tester
> will find significant defects. In XP, he/she would endeavor to find these
> defects as soon as possible, maybe by clarifying user stories or improving
> developer tests.
>
Developer tests merely express what the developer thinks each unit of code
should do.  If the developers are doing TDD, I would expect exploratory
testing to find few defects that were because the code did something
different than the developers thought it should.  I would expect most of the

defects that an exploratory tester would find would be sequences of events
that nobody had considered when creating the customer tests.  I good way to
think of these defects are missing requirements.

> However, I still expect that the tester would find significant defects
> during/latter part of the iteration. I think that would require
> creative/analytical/thinking skills that XP practices can't guarantee.
>
There are no guarantees no matter how software is developed.  There are no
silver bullets.  Agile is about honing in on perfection by reflecting and
(Continue reading)

Permalink | Reply | 0 comments |
headers
Ron Jeffries | 2 Jun 2010 02:18
Picon
Favicon
Gravatar

Re: Re: Shouldnt done include everything.

Hello, xtremenilanjan.  On Tuesday, June 1, 2010, at 10:18:17 AM,
you wrote:

> This is aside from my original question-  

> Isn't it setting the bar high - if we expect programmers to keep
> programming till the end of the iteration? :-) Do let me know if I
> should expect this.

Yes, it's setting the bar high. How good do you want to be?

> I am thinking more from the point of view of a software product
> team

Me too.

Ron Jeffries
www.XProgramming.com
www.xprogramming.com/blog
If not now, when?  -- The Talmud
Permalink | Reply | 40 comments |
headers
Ron Jeffries | 2 Jun 2010 02:23
Picon
Favicon
Gravatar

Re: Re: Shouldnt done include everything.

Hello, xtremenilanjan.  On Tuesday, June 1, 2010, at 10:40:24 AM,
you wrote:

> My context: Developing software products/shrink wrapped

Yes.

> Despite all XP practices, I would expect a good
> tester/exploratory tester will find significant defects.

If she does, then the programmers aren't doing a very good job of
testing, and the team hasn't done a very good job of eliciting
requirements and automating enough acceptance tests to be sure that
things work and stay working.

Significant defects in exploration is prima facie evidence of bad
work. That's what needs to be fixed.

> In XP,
> he/she would endeavor to find these defects as soon as possible,
> maybe by clarifying user stories or improving developer tests.

If a test is written BEFORE the code, it doesn't detect defects, it
prevents them.

> However, I still expect that the tester would find significant
> defects during/latter part of the iteration.

If so, the team is not performing well enough yet.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Bill Caputo | 2 Jun 2010 03:10
Favicon

Re: Call for Short Papers and Posters, ESEM 2010, Empirical Software Engineering and Measurement

On Thu, May 20, 2010 at 4:43 AM, Succi Giancarlo (P)
<Giancarlo.Succi <at> unibz.it> wrote:
> ** APOLOGIZE IF YOU RECEIVE MULTIPLE COPIES**

I'm confused - why should I have to apologize if I receive multiple copies?

Bill
Permalink | Reply | 0 comments |
headers
JackM | 2 Jun 2010 23:35
Favicon

Re: Shouldnt done include everything.

While I agree that setting the bar high is good. I don't believe realistically that if you stop coding on the
last day that you're Done on the last day. 

Not the Done in my books. 

Until you have integrated all the code on that last day and regression tested all the aspects of the code that
was changed I don't believe you're done done.

So in my opinion you have to plan on being code complete and unit test complete at least a couple days prior to
end of sprint. That leaves time for exploratory testing you're talking about and time to resolve any
issues prior to end of sprint.

Jack
blog.agilebuddy.com
twitter.com/agilebuddy
www.agilebuddy.com

--- In extremeprogramming <at> yahoogroups.com, Ron Jeffries <ronjeffries <at> ...> wrote:
>
> Hello, xtremenilanjan.  On Tuesday, June 1, 2010, at 10:18:17 AM,
> you wrote:
> 
> > This is aside from my original question-  
> 
> > Isn't it setting the bar high - if we expect programmers to keep
> > programming till the end of the iteration? :-) Do let me know if I
> > should expect this.
> 
> Yes, it's setting the bar high. How good do you want to be?
>
(Continue reading)

Permalink | Reply | 39 comments |

Gmane