Thursday, January 07, 2010
« Code Mastery event in Boston: Feb 11 | Main | Spaghetti Podcast online »
Random unit test failures are a pain…

I’ve been working in the VS10 beta trying to get my unit tests converted from nunit to mstest. Overall that went very smooth and easy thanks to some great foresight on the part of the Magenic guys who put together most of the tests.

But I keep running into issues where random tests fail due to a file IO exception:

“a procedure imported by System.Transactions could not be loaded”

The same tests may fail or work seemingly at random. After chasing this far too long, I think I’m going to assume it is a .NET 4 beta bug and just live with it for now. I can’t see where it is my code, since this code is unchanged from .NET 3.5 to 4.0, and any given test will pass around 90% of the time and fail around 10% of the time – and they only fail when run in groups of tests – so I strongly suspect something is messed up in the .NET type loader or something along that line…

Frustrating though – hopefully the next release of VS10 will magically resolve the issue.

CSLA .NET
Thursday, January 07, 2010 12:05:01 PM (Central Standard Time, UTC-06:00)  #    Disclaimer  |  Comments [6]  |  Related posts:
CSLA 4 version 4.3.0 alpha available
Using CSLA 4: Windows Phone ebook available
CSLA 4 version 4.2 released
Improving CSLA 4 MobileFormatter behavior
Silverlight 6 doesn’t matter
DevCon slides
Thursday, January 07, 2010 1:25:46 PM (Central Standard Time, UTC-06:00)
Oddly enough I have seen similar things even under VS2008. We had unit tests that used transactions that would fail if the transaction was part of the test method but if we moved it out to the test setup/tear-down then the tests would work without issue. These issues only showed up when running all the tests at once as part of automated testing. Even after posts to the Microsoft forums about this where I was informed that the tests are run sequentially and that one will not start until after the last has finished, I think it has to do with a timing issue of the transaction cleaning up after itself before the next one is created.
Adam
Thursday, January 07, 2010 1:50:17 PM (Central Standard Time, UTC-06:00)
I found the following on stackoverflow -- may or may not be helpful:

http://stackoverflow.com/questions/1945589/a-procedure-imported-by-myassembly-could-not-be-loaded

The random failures sure do make it seem like a timing problem. Would it be worth trying to put a delay into ClassInitialize to make sure that the file I/O's have had a chance to settle down before the tests start running?
D. Lambert
Thursday, January 07, 2010 2:36:40 PM (Central Standard Time, UTC-06:00)
I'm beginning to think maybe I should have stayed with nunit :)
Rockford Lhotka
Friday, January 08, 2010 11:01:10 AM (Central Standard Time, UTC-06:00)
It sounds like a threading issue. Also could it have anything to do with object destruction? I know the GC is different in .net 4. Were you running these in nunit successfully on .net 4 before migrating to mstest?
Justin Chase
Friday, January 08, 2010 11:22:40 AM (Central Standard Time, UTC-06:00)
I just discovered that VS10 throws this same exception if I try to build the project in release mode. Actually SGEN is what fails - but it is in the VS build process.

So now I suspect the VS10 upgrade wizard may not have done the right thing when upgrading my 3.5 project to 4.0...
Rockford Lhotka
Saturday, January 09, 2010 1:28:16 PM (Central Standard Time, UTC-06:00)
More clues - it appears now to be a VS10 install issue, as in "VS10 didn't install correctly". I've tried to repair but that made it worse. I'm doing an uninstall/reinstall now, but I think I'm glad I installed the beta in a virtual machine!!
Rockford Lhotka
Comments are closed.