Test-first JNI experiences
Robert Wenner <robert <at> port25.com>
2002-09-04 12:26:25 GMT
I was curious to see whether a Java API using JNI to access a C API
can be done test-first. Here are some notes on what I did. I am sorry,
this is quite long.
Make AccounterTest.java. Test fails, no Accounter.class.
Make Accounter.java with empty constructor -> test passes.
Leave constructor empty, look at first method to implement, which is
Make a test to open an accounting file. Assuming the C API is my
user story ("must work as in C") I look at the C function and
what arguments it takes: file name and unsigned int options. I define
static final ints for the possible option values. I document the new
method by recycling and adjusting the documentation from the C code.
Return codes from C are supposed to be Exceptions in Java, so I have a
throws AccounterException specification.
As I do not know where accounting files are stored I try to open
an non-existing file and want to catch an AccounterException for this.
Does not compile, so I add an open method to Accounter.
Still does not compile, no AccounterException defined.
Make a simple exception class, AccounterException.
Tests compile, but new test fails: no exception is thrown.
Have the open method always throw an AccounterException.
Make another test for open, this time on an existing file. Test fails.
I guess now I really have to call the C part.
I consider replacing the C part with dummies, but the Java API is
supposed to just hand over everything to C, so it does not have
responsibility at all. Any error checking is also done in C.
Replace throw with call to native code in open method.
Again, test fails, this time with unsatisfied linker errors: the native
part does not exist yet.
Create native part from generated header, methods don't do anything yet,
not even call the C routines. (We have another layer between the Java
and the C API where error checking and turning return values into
Exceptions is done. This layer is also required because JNI dictates the
names the native functions must have.)
Now the linker errors are gone but the test to open an existing file
(still / again) fails.
Now I add real calls to the C API in my layer code. Works.
At this point there seems to be no more sense in TFD, as all I do
is writing a test case and then passing given arguments to the C API.
I guess my Java methods are to simple to break.
Two hours later requirements have changed, there needs to be an
implicit destroy method, like calling the native context's destructor.
This is for freeing native resources the other API holds like memory
and a file handle. For performance reasons and to avoid a bottleneck we
do not want to wait for the finalizer being called sometime.
This changes my code a little, the Java API code has to make
sure the underlying native context is still valid or throw an
IllegalStateException. Writing that test is much more interesting
I write a test to access a destroyed native context and after it fails
I implement a check in the method to return the context (getContext)
that is called before each native call. Now accessing destroyed
native objects results in a IllegalStateException. All tests pass.
I continue next day writing a new test on read some data from an
opened file. Reading is tricky: the API I have to use defines
callbacks and while reading will call the functions one has
registered earlier. I guess in Java this would be a method in
my JNI Accounter class which is supposed to be overwritten by
subclasses, like in the Template Method pattern, and I will provide
a default implementation that does nothing.
The tests now use a sub-classed Accounter (DummyAccounter) that
counts how often a callback handler is called and checks whether
these calls are correct, i.e. whether in the right order and for
corresponding. (The calls are beginStructure, processField, and
endStructure, so I can assert there are no more end calls than
beginCalls and processField occurs only during processing a
My tests fail, since my C code does not yet call back into Java.
I write these callbacks. I copy the Java context and store it
in a void pointer called user data. The C API allows setting the
user data and will provided it when doing the callbacks. I need
this to know for which Java object the callback is.
The whole program crashes with signal 11, segmentation violation.
These JNI functions are hard to debug and it takes me some hours
to find out I can not store an jobject between two JNI calls, as
it may be relocated by the garbage collector. I move registering
my callback handlers from object construction to the read call and
I add further tests to see whether my callback methods in Java
are handed the correct arguments. This should not be tested, since
the data comes from the C API, but since I transform a key / value
pair string array into a HashMap I assume some testing won't hurt.
My test fails, since I did not write anything to the list.
To make the test pass I write fixed values into the HashMap, exactly
one key / value pair. The test passes and I retrieve my value by
looking up the key, no more, no less.
I add another test to expect more than one items in the HashMap.
It fails and I add three items (also) hard-coded. This does not break
the previous test, since beginStructure as well as processField accept
Now I want to get rid of the hard coded testing stuff in my code.
I write a C program as a dummy C API. It returns fixed values
(my hard coded values from earlier tests) and for testing I just
have to make sure the linker finds my dummy library before the real
one, i.e. I have to set LD_LIBRARY_PATH to contain the dummy library's
directory at first position. I put that in the Makefile.
After this refactoring my tests still work.
Next I want to refactor the code that does the callbacks. There are lots
of duplication here, each time lookup the class id, the method ids for
Accounter object, the class ids for the HashMap, the HashMap constructor
method id, the HashMap.put method id... I move the lookup in one place,
in the read method and store the class and method ids in the user data
I already carry around.
Still all tests pass and I am done.
Bottom line: I suggest doing JNI test first, even if you just have
wrappers. A little typo on a method or class name can crash the program.
Storing pointers to long can crash the program. Using the wrong
library can crash the program. These crashes (segmentation violations)
are not caught by JUnit, still finding them is valuable.