Re: NUnit migration and test failure status
Atsushi Eno <atsushi <at> ximian.com>
2009-01-05 03:58:58 GMT
We still have a *lot* of test failures in our unit tests, and I'd like
to give some updates on analysis.
(Note: I didn't walk through every failure. Also my build is not always
clean. Verify it at your side before reading them blindly.)
- System.Threading.ExecutionContextTest.Copy_FromThread:
it does not fail on .NET, so it's likely a bug in runtime or corlib.
- System.Data.DataColumnTest.ExpressionAggregates:
it is rather likely due to nunit change with related to current
culture handling. This test does not fail when I give TESTNAME or
FIXTURE option at run-test.
- System.Web.UI.WebControls.MenuTest.Menu_DefaultRender:
similarly it does not fail when I run only this test (with TESTNAME).
- Mono.Data.Sqlite.SqliteCommandUnitTests.Select:
I copied Mono.Data.Sqlite.dll to mcs/class/Mono.Data.Sqlite directory
and did run-test-ondotnet. The result was the same failure as on mono,
so I assume it is rather the test case that is wrong.
- the same applies to Mono.Data.SqliteClient.
- System.Runtime.Remoting: we have no remedy. We will have to either
fully disable the tests for this assembly or revert NUnit to 2.2.
FWIW they fail on .NET either.
- Microsoft.Build.Tasks
.CreateCSharpManifestResourceNameTest.TestNoRootNamespaceNoCulture:
It does not fail on mono/w32. Instead some tests in the assembly do
not run successfully on mono/w32. Another set of tests fail on .NET.
- System.Transactions.AsyncTest.AsyncFail3:
It doesn't fail on mono/w32 either.
- System.Web.UI.WebControls.ListViewTest.ListView_Edit:
It doesn't, either.
Our nunit fails to handle System.Web_test_net_2_0.dll under .NET with
nunit-console, so I cannot verify if those sys.web tests are fine.
Thanks to Marek (grendel), the floating point issues are gone.
Also thanks to Sebastien, the drawing tests are fixed to pass.
Atsushi Eno
Atsushi Eno wrote:
> Hello,
>
> As some of you may have noticed, I have upgraded our local use of nunit
> from 2.2.0 to 2.4.8. It involved some changes in our test run, but
> should be invisble to most of you.
>
> The build has been broken for a while (am sorry for that; it was a set
> of make dist / packaging mess), and now we have a couple of test
> failures. So before Marc dives into the hell list of unknown bugs to
> him, I wanted to give some survey on them:
>
> - some JIT stuff: test-verifier, test-System_Web-2.0
> - mere build failure: test-System_Data-2.0
> - some floating point failures: test-corlib-*, test-System_Drawing-*,
> test-System_Data-1.0, test-System_Runtime_SFS-*
> - possible NUnit run changes: test-corlib-2.0
> (MonoTests.System.Threading.ExecutionContextTest.Copy_FromThread),
> test-System_Runtime_Remoting-*
> - possibly gdiplus: test-System_Drawing-*, test-Windows_Forms-2.0
> (MonoTests.System.Drawing.PenTest.DashCap_Valid)
> - some sys.data changes: test-Mono_Data_Sqlite-*,
> test-Mono_Data_SqliteClient*
>
> The only pattern I am concerned is the one with (possible) NUnit run
> changes. NUnit 2.4.8 now requires -noshadow to get our tests run fine.
> This also happens under .NET (i.e. make run-test-ondotnet) even without
> -noshadow, so I am rather afraid that NUnit 2.4.8 unlike 2.2.0 cannot
> handle it.
>
> I'd like to hear inputs on what could be done to fix this issue.
>
> Atsushi Eno
> _______________________________________________
> Mono-devel-list mailing list
> Mono-devel-list <at> lists.ximian.com
> http://lists.ximian.com/mailman/listinfo/mono-devel-list
>
RSS Feed