Re: TDD is about developing not testing (was Unit Test Code Generation)

On 22 Aug 2002, Dave Astels wrote:

> Comments?  Is it too late to ditch the T in TDD?  What could it be
> called?

How about just dropping the T?  In Extreme Programming we use Driven
Design.  It has that extreme feel to it.

RE: TDD is about developing not testing (was Unit Test Code Generation)

Re: TDD is about developing not testing (was Unit Test Code Generation)

Around Tuesday, September 3, 2002, 4:02:47 PM, Brian C. Robinson wrote:

> How about just dropping the T?  In Extreme Programming we use Driven
> Design.  It has that extreme feel to it.

;->   but ...

The horse has left the building, guys.

Ron Jeffries
www.XProgramming.com

First they ignore you, then they laugh at you, then they fight you, then you win.
  - Gandhi
------------------------ Yahoo! Groups Sponsor ---------------------~-->
4 DVDs Free +s&p Join Now
http://us.click.yahoo.com/pt6YBB/NXiEAA/MVfIAA/NhFolB/TM
---------------------------------------------------------------------~->

To unsubscribe from this group, send an email to:
testdrivendevelopment-unsubscribe <at> yahoogroups.com

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 

Re: TDD is about developing not testing (was Unit Test Code Generation)

Hello Mark,

Tuesday, September 03, 2002, 9:33:41 PM, you wrote:
> I have to agree with that one.  I've never really agreed with the "Testing"
> part (and I don't want to get into it again!!!). Driven Design sounds
> exactly like what is being done.  How that "driving" is done: testing,
> examples, samples, what-ever... is up to the XP team to name it, but I think
> we are all agreed on HOW its done.  So, kudos to Brian!

What about Model Driven Design? Is that compatible with XP? So if I
make a model in UML and than starts to implement it, I can say I use
Driven Design? The model drives the design just as tests do.

I think we better stick with TDD, I like it! (what more reason do you
need  )

--

-- 
Best regards,
 Jonas

Education is the ability to listen to almost anything without losing
your temper.
 Robert Frost

Test-first JNI experiences

I was curious to see whether a Java API using JNI to access a C API
can be done test-first. Here are some notes on what I did. I am sorry, 
this is quite long.

Make AccounterTest.java. Test fails, no Accounter.class.

Make Accounter.java with empty constructor -> test passes.

Leave constructor empty, look at first method to implement, which is 
Open.

Make a test to open an accounting file. Assuming the C API is my
user story ("must work as in C") I look at the C function and
what arguments it takes: file name and unsigned int options. I define
static final ints for the possible option values. I document the new
method by recycling and adjusting the documentation from the C code.
Return codes from C are supposed to be Exceptions in Java, so I have a
throws AccounterException specification.

As I do not know where accounting files are stored I try to open
an non-existing file and want to catch an AccounterException for this.

Does not compile, so I add an open method to Accounter. 

Still does not compile, no AccounterException defined.
Make a simple exception class, AccounterException.

Tests compile, but new test fails: no exception is thrown.

Have the open method always throw an AccounterException. 
Test passes.

Make another test for open, this time on an existing file. Test fails.
I guess now I really have to call the C part.

I consider replacing the C part with dummies, but the Java API is 
supposed to just hand over everything to C, so it does not have 
responsibility at all. Any error checking is also done in C.

Replace throw with call to native code in open method.

Again, test fails, this time with unsatisfied linker errors: the native
part does not exist yet.

Create native part from generated header, methods don't do anything yet,
not even call the C routines. (We have another layer between the Java
and the C API where error checking and turning return values into
Exceptions is done. This layer is also required because JNI dictates the
names the native functions must have.)

Now the linker errors are gone but the test to open an existing file
(still / again) fails.

Now I add real calls to the C API in my layer code. Works.

At this point there seems to be no more sense in TFD, as all I do
is writing a test case and then passing given arguments to the C API.
I guess my Java methods are to simple to break.

Two hours later requirements have changed, there needs to be an
implicit destroy method, like calling the native context's destructor.
This is for freeing native resources the other API holds like memory 
and a file handle. For performance reasons and to avoid a bottleneck we 
do not want to wait for the finalizer being called sometime.

This changes my code a little, the Java API code has to make
sure the underlying native context is still valid or throw an 
IllegalStateException. Writing that test is much more interesting 

I write a test to access a destroyed native context and after it fails
I implement a check in the method to return the context (getContext)
that is called before each native call. Now accessing destroyed
native objects results in a IllegalStateException. All tests pass.

I continue next day writing a new test on read some data from an
opened file. Reading is tricky: the API I have to use defines 
callbacks and while reading will call the functions one has
registered earlier. I guess in Java this would be a method in
my JNI Accounter class which is supposed to be overwritten by 
subclasses, like in the Template Method pattern, and I will provide
a default implementation that does nothing.

The tests now use a sub-classed Accounter (DummyAccounter) that 
counts how often a callback handler is called and checks whether
these calls are correct, i.e. whether in the right order and for
corresponding. (The calls are beginStructure, processField, and
endStructure, so I can assert there are no more end calls than
beginCalls and processField occurs only during processing a 
structure.)

My tests fail, since my C code does not yet call back into Java.

I write these callbacks. I copy the Java context and store it
in a void pointer called user data. The C API allows setting the
user data and will provided it when doing the callbacks. I need 
this to know for which Java object the callback is.

The whole program crashes with signal 11, segmentation violation.
These JNI functions are hard to debug and it takes me some hours
to find out I can not store an jobject between two JNI calls, as
it may be relocated by the garbage collector. I move registering
my callback handlers from object construction to the read call and
it works.

I add further tests to see whether my callback methods in Java
are handed the correct arguments. This should not be tested, since
the data comes from the C API, but since I transform a key / value
pair string array into a HashMap I assume some testing won't hurt.

My test fails, since I did not write anything to the list.

To make the test pass I write fixed values into the HashMap, exactly
one key / value pair. The test passes and I retrieve my value by
looking up the key, no more, no less.

I add another test to expect more than one items in the HashMap.
It fails and I add three items (also) hard-coded. This does not break
the previous test, since beginStructure as well as processField accept
a HashMap.

Now I want to get rid of the hard coded testing stuff in my code.
I write a C program as a dummy C API. It returns fixed values
(my hard coded values from earlier tests) and for testing I just
have to make sure the linker finds my dummy library before the real 
one, i.e. I have to set LD_LIBRARY_PATH to contain the dummy library's
directory at first position. I put that in the Makefile.

After this refactoring my tests still work.

Next I want to refactor the code that does the callbacks. There are lots
of duplication here, each time lookup the class id, the method ids for
Accounter object, the class ids for the HashMap, the HashMap constructor
method id, the HashMap.put method id... I move the lookup in one place,
in the read method and store the class and method ids in the user data
I already carry around.

Still all tests pass and I am done.

Bottom line: I suggest doing JNI test first, even if you just have
wrappers. A little typo on a method or class name can crash the program.
Storing pointers to long can crash the program. Using the wrong
library can crash the program. These crashes (segmentation violations) 
are not caught by JUnit, still finding them is valuable.

Robert

methods for testing string char memory leaks

Hi,

I'm pretty new to test first development, but am beginning to enjoy it, and
have already experienced the benefits of the tests extending one's
imagination.

I have one current testing issue.

The C++ project I'm working on interfaces with some classes that a couple of
other programmers have written. They have not used xunit type testing before
and I see it as an opportunity to get them to see the benefits and start to
use it too.

The issue I have is that I have identified a couple of problems with these
other classes and would like to write some tests to demonstrate the bugs.
However the bugs relate to memory leaks.

One actual problem appears to be the way an fstream is fed a filename as a
string, when I run my tests on the class it works fine as far as my current
tests are concerned, but in production it sometimes produces filenames that
have junk appended to the end. This only occurs in the production program,
not when I run my cppunit tests, as in production more memory surrounding
the string is being exercised.

their code is something like:

int CFileWrapper::open(string fn)
{
  fd.open(fn.data(), ios::in|ios::out|ios::app);  // open filestream

Does anyone have any testing methods that address memory leak issues with
strings or char arrays that can exercise the memory around a given string to
check for memory leaks.

In this case the file could be used for reading or writing so just testing
if the file exists is not a complete solution.

Thank in advance for any paradigm shifting thoughts......

Regards,
Phil.

Re: methods for testing string char memory leaks

on 2002.09.07 3:12 PM, philp9 at philp9 <at> yahoo.com wrote:

> Hi,
> 
> I'm pretty new to test first development, but am beginning to enjoy it, and
> have already experienced the benefits of the tests extending one's
> imagination.
> 
> I have one current testing issue.
> 
> The C++ project I'm working on interfaces with some classes that a couple of
> other programmers have written. They have not used xunit type testing before
> and I see it as an opportunity to get them to see the benefits and start to
> use it too.
> 
> The issue I have is that I have identified a couple of problems with these
> other classes and would like to write some tests to demonstrate the bugs.
> However the bugs relate to memory leaks.
[...]
> Does anyone have any testing methods that address memory leak issues with
> strings or char arrays that can exercise the memory around a given string to
> check for memory leaks.

Metrowerks CodeWarrior ships with DebugNew.h / DebugNew.cpp, which
re-implements global "new" and "delete" with memory-leak checking and other
debugging aids... Ideally, they'd also re-implement "malloc" and "free", and
all other OS-specific functions that allocate memroy... I use this on MacOS.

Using Metrowerks' code, you can use the macro NEW instead of "new" to record
the filename and linenumber of each allocation -- very helpful in tracking
down memory leaks.

They provide a function to call to clear the memory-leak record, and another
function to check for memory leaks, as well as one to save a log file
listing the memory leaks. My unit test runner calls these functions, and all
of my code uses the macro "NEW", so I've gotten rid of a lot of obvious
memory leaks. Unfortunately, there are still LOTS of small memory leaks, and
they don't have filename/line-number information... these leaks are
apparently from mis-using the C++ standard library classes.

I should modify my test-runner to clear the record and check for leaks
before and after each unit test, to help me identify the less-obvious leaks
from mis-using the C++ standard library classes...

On Windows, we have BoundsChecker, which can detect memory leaks and lots of
other bad things.

----

C. Keith Ray
<http://homepage.mac.com/keithray/resume2.html>
<http://homepage.mac.com/keithray/xpminifaq.html>

Re: methods for testing string char memory leaks

Phil,

Part of your problem relates to how C++ works in a debug mode versus release
mode and part of it relates to your code.  To convert a C++ string into a C
style string terminated by a null character, you need to use the c_str()
method, not the data() method.  The c_str() method puts a null at the end of
the string in the data buffer and then returns the pointer to the data
buffer.  The data() method simply returns the pointer to the data buffer
without putting in the null/zero byte at the end.  The reason your code
works in debug mode but not in release mode is that in debug mode memory is
initialized to zeros (automatically giving you a null bytes in the buffer).
But in release mode, memory is simply left uninitialized (leaving you with
the extraneous characters you see).

You might or might not still have any memory leaks, but first write a test
verifying your C++ string to C style string conversion has the correct
termination character for C style strings.

Regards,
Jim Murphy
murphyjr <at> mags.net

Re: methods for testing string char memory leaks

Jim,

Thanks, I had figured out what was the cause of the error, but it 
wasn't my code to change. I had come to the same conclusion of how to 
fix it as you did.

What I was after was a way to deliver a test that would highlight and 
expose the symptoms without specifically saying how to fix it. The 
testing method would need to uncover these type of issues and I was 
hoping for someone to have a novel way of exposing these types of 
errors especially were they may be more obsure.

I note your suggestion about verifying your C++ string to C style 
string conversion has the correct termination character for C style 
strings. However to test the class using black box testing I can only 
provide a filename string as an argument to the wrapper class. The 
code I showed was the classes internal private code (the root cause). 
The only external class symptom was some strange filenames appearing 
the directory where the file was to be created or read from in 
production. These strange files didn't appear while running the 
cppunit tests as previously noted.

Any ideas on how to create an external test to detect this type of 
bug?

Regards,
Phil.

--- In testdrivendevelopment <at> y..., "Jim Murphy" <murphyjr <at> m...> wrote:
> Phil,
> 
> Part of your problem relates to how C++ works in a debug mode 
versus release
> mode and part of it relates to your code.  To convert a C++ string 
into a C
> style string terminated by a null character, you need to use the 
c_str()
> method, not the data() method.  The c_str() method puts a null at 
the end of
> the string in the data buffer and then returns the pointer to the 
data
> buffer.  The data() method simply returns the pointer to the data 
buffer
> without putting in the null/zero byte at the end.  The reason your 
code
> works in debug mode but not in release mode is that in debug mode 
memory is
> initialized to zeros (automatically giving you a null bytes in the 
buffer).
> But in release mode, memory is simply left uninitialized (leaving 
you with
> the extraneous characters you see).
> 
> You might or might not still have any memory leaks, but first write 
a test
> verifying your C++ string to C style string conversion has the 
correct
> termination character for C style strings.
> 
> 
> Regards,
> Jim Murphy
> murphyjr <at> m...

Re: methods for testing string char memory leaks

Hi Keith,

Thanks for your input.

I note your suggestion to clear the record and check for leaks
before and after each unit test, however as the code shown was 
internal private class code, the black box testing only knows about 
the original string. It's a bit hard to check how the class 
internally converts the string from outside the class as well as the 
fact it only seems to happen when in production when memory is 
exercised more randomly.

Any ideas or strategies?

Regards,
Phil.

--- In testdrivendevelopment <at> y..., "C. Keith Ray" <ckeithray <at> a...> 
wrote:
> on 2002.09.07 3:12 PM, philp9 at philp9 <at> y... wrote:
> 
> > Hi,
> > 
> > I'm pretty new to test first development, but am beginning to 
enjoy it, and
> > have already experienced the benefits of the tests extending one's
> > imagination.
> > 
> > I have one current testing issue.
> > 
> > The C++ project I'm working on interfaces with some classes that 
a couple of
> > other programmers have written. They have not used xunit type 
testing before
> > and I see it as an opportunity to get them to see the benefits 
and start to
> > use it too.
> > 
> > The issue I have is that I have identified a couple of problems 
with these
> > other classes and would like to write some tests to demonstrate 
the bugs.
> > However the bugs relate to memory leaks.
> [...]
> > Does anyone have any testing methods that address memory leak 
issues with
> > strings or char arrays that can exercise the memory around a 
given string to
> > check for memory leaks.
> 
> Metrowerks CodeWarrior ships with DebugNew.h / DebugNew.cpp, which
> re-implements global "new" and "delete" with memory-leak checking 
and other
> debugging aids... Ideally, they'd also re-implement "malloc" 
and "free", and
> all other OS-specific functions that allocate memroy... I use this 
on MacOS.
> 
> Using Metrowerks' code, you can use the macro NEW instead of "new" 
to record
> the filename and linenumber of each allocation -- very helpful in 
tracking
> down memory leaks.
> 
> They provide a function to call to clear the memory-leak record, 
and another
> function to check for memory leaks, as well as one to save a log 
file
> listing the memory leaks. My unit test runner calls these 
functions, and all
> of my code uses the macro "NEW", so I've gotten rid of a lot of 
obvious
> memory leaks. Unfortunately, there are still LOTS of small memory 
leaks, and
> they don't have filename/line-number information... these leaks are
> apparently from mis-using the C++ standard library classes.
> 
> I should modify my test-runner to clear the record and check for 
leaks
> before and after each unit test, to help me identify the less-
obvious leaks
> from mis-using the C++ standard library classes...
> 
> On Windows, we have BoundsChecker, which can detect memory leaks 
and lots of
> other bad things.
> 
> ----
> 
> C. Keith Ray
> <http://homepage.mac.com/keithray/resume2.html>
> <http://homepage.mac.com/keithray/xpminifaq.html>