Makiwa

I fancy a bit of rant. And today’s punchbag is Visual Studio’s Unit Test Wizard.

It is evil and it should be dropped from the product.

My previous company outsourced the development of a software project and I ended up reviewing the code from the first iteration. I had concerns about some of the code; one concern was that they had used Visual Studio’s Unit Test Wizard to auto-generate their unit tests.

There are three main reasons I dislike this feature.

1. Discourages test-driven development
   There is a growing consensus that test-driven development results in better tests, better code coverage, higher quality code, and more productive development. Visual Studio’s auto-generation of test methods discourages such a test-driven approach, as it requires the code to have been written before the tests are generated.
2. Causes method-centric tests
   Auto-generation of test methods results in tests that are method-centric rather than requirement-centric. The issue with auto-generated, method-centric tests is that a) they don’t have descriptive names, and b) they promote the belief that if you write a test that calls that method then it is “covered” - regardless of how many branches there are in the logic of the method.
3. Creates homogenous tests
   I believe that the unit tests should be a collection of “little stories” that describe the system. But auto-generated tests all look the same. They don’t give you a good feel for what a system does because each one is just the setup for a method call and the evaluation of the result. To paraphrase Groove Armada, if every test looks the same, you get tired of looking at them!
4. Causes test fatigue and weak tests
   This is perhaps the biggest problem with the tool and it results from not using a test-driven approach. Once Visual Studio has kindly generated, say, 50 test method stubs, the developer is faced with the task of “filling them all in”. In my opinion, these tests are likely to be weak. Not only because the developer is temporally disconnected from the spirit and nuances of the various underlying requirements than he would have been with a test-first approach. But he now has a mammoth task ahead of him. By the time he hits test #15 he is going to be suffering from “test fatigue” and is very likely to start compromising on test quality to just get it over and done with.

I’m not saying that ALL unit tests developed with this wizard are necessarily flawed. I’m just saying that it increases the liklihood of poor quality tests. And poor quality tests are perhaps more dangerous than no tests…”I ran the Unit Test Wizard, filled in the blanks, did some refactoring. Whadya know! All the tests pass…RELEASE IT!

I’d like to consign this feature to the dustbin.

This entry was posted on Sunday, July 16th, 2006 at 23:44 and is filed under Agile Development, Development, Extreme Programming, Visual Studio. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.