My previous company outsourced the development of a software project and I ended up reviewing the code from the first iteration. I had concerns about some of the code; one concern was that they had used Visual Studio’s Unit Test Wizard to auto-generate their unit tests.

There are three main reasons I dislike this feature.

1. Discourages test-driven development
   There is a growing consensus that test-driven development results in better tests, better code coverage, higher quality code, and more productive development. Visual Studio’s auto-generation of test methods discourages such a test-driven approach, as it requires the code to have been written before the tests are generated.
2. Causes method-centric tests
   Auto-generation of test methods results in tests that are method-centric rather than requirement-centric. The issue with auto-generated, method-centric tests is that a) they don’t have descriptive names, and b) they promote the belief that if you write a test that calls that method then it is “covered” - regardless of how many branches there are in the logic of the method.
3. Creates homogenous tests
   I believe that the unit tests should be a collection of “little stories” that describe the system. But auto-generated tests all look the same. They don’t give you a good feel for what a system does because each one is just the setup for a method call and the evaluation of the result. To paraphrase Groove Armada, if every test looks the same, you get tired of looking at them!
4. Causes test fatigue and weak tests
   This is perhaps the biggest problem with the tool and it results from not using a test-driven approach. Once Visual Studio has kindly generated, say, 50 test method stubs, the developer is faced with the task of “filling them all in”. In my opinion, these tests are likely to be weak. Not only because the developer is temporally disconnected from the spirit and nuances of the various underlying requirements than he would have been with a test-first approach. But he now has a mammoth task ahead of him. By the time he hits test #15 he is going to be suffering from “test fatigue” and is very likely to start compromising on test quality to just get it over and done with.

I’m not saying that ALL unit tests developed with this wizard are necessarily flawed. I’m just saying that it increases the liklihood of poor quality tests. And poor quality tests are perhaps more dangerous than no tests…”I ran the Unit Test Wizard, filled in the blanks, did some refactoring. Whadya know! All the tests pass…RELEASE IT!

I’d like to consign this feature to the dustbin.

After years of being disparaged by some in the software development community, the waterfall process is back with a vengeance. You’ve always known a good waterfall-based process is the right way to develop software projects. Come to the Waterfall 2006 conference and see how a sequential development process can benefit your next project. Learn how slow, deliberate handoffs (with signatures!) between groups can slow the rate of change on any project so that development teams have more time to spend on anticipating user needs through big, upfront design.

Heheheh

The solution was to add the following line to the <startup> tag of nunit-gui.exe.config (and nunit-console.exe.config if you are using it from the command line):

EDIT: For the final release of Visual Studio 2005 you would of course use:

You may need to comment out all the other supportedRuntime entries to get it to work.

Having got NUnit to test a .NET v2.0 assembly, my next task was to get it working as the debug harness for my class library projects like I did with Visual Studio 2003.

I was quite hopeful. I did everything as I had done for my VS 2003 projects…but no - they NUnit and VS2005 did not want to play nicely together! When I ran the project, NUnit did launch, but when I clicked “Run” nothing happened.

I spend a couple of hours trying to get it to work but to no avail. Eventually, out of desperation I downloaded the latest iteration build of NUnit (version 2.2.2 as of this writing). I set the 2.2.2 version of nunit-console.exe to be the startup program (remembering to update its config file with the “supportedRuntime” stuff mentioned above)…and voilà! It works a treat!

So if you can’t get NUnit to work with VS 2005 - try to download the latest drop from http://www.nunit.org/download.html

My main requirements were that:

1. It should be version control friendly (shouldn’t require additional setup steps after checking it out from version control – it should just work).
2. You should be able to debug the class library by setting one or more breakpoints and then just running a test.

Fortunately, perseverance has triumphed over ineptitude and I have come up with a solution.

What follows is a step-by-step guide to setting it all up.

1. Install NUnit
   I am currently using NUnit 2.2. Download a copy from the NUnit website (www.nunit.org) and install it.
2. Create a blank solution
   By creating a blank solution first and then adding the project, you don’t end up with the solution and project files in the same place. This isn’t critical but it helps if you ever add other projects. That way your solution file will be in the logical place - the folder above all of the projects.

   I created Solution1.sln in a folder called Solution1.

3. Add a new Class Library project to that solution.

   

4. Add a test class
   Add a new class called Class1Tests.

   

5. Add a reference to NUnit framework

   

6. Add a new test
   In Class1Tests.cs you need to add a using statement (NUnit.Framework).
   You should then add a [TestFixture] attribute to the class.
   Now create a stub test method: CreateClass1. This new method needs a [Test] attribute.

   You can now build the project.

   

7. Open the class library in NUnit
   Run up NUnit and click File | Open… and select your newly build class library DLL.

   

   You can run the test if you like - it should pass.

   

8. Save the NUnit project
   Now you need to save the NUnit project. The NUnit project file is an XML file (surprise) that points to the assemblies that NUnit should load.

   Click File Save As…, but don’t save it in the Debug folder. Rather go up a couple of levels and save it in the project folder. Call it ClassLibarary1.nunit.

   

9. Setup NUnit as the class library’s startup process
   Back in Visual Studio, select the ClassLibrary1 project in the Solution Explorer.
   Right-click and choose Properties.

   Expand “Configuration Properties” and then select “Debugging”.

   

   Set “Debug Mode” to “Program”. Then click Apply.

   

   Set “Start Application” to the path to nunit-gui.exe. The default is “C:\Program Files\NUnit 2.2\bin\nunit-gui.exe”.

   

   Set the Command Line Arguments to the relative path to the NUnit project file (the ClassLibrary1.nunit file we saved in the previous step).
   This should be “..\..\ClassLibrary1.nunit” (the relative path from \ClassLibrary1\bin\Debug, to \ClassLibrary1).

   

   At this point, if you hit F5 to run the project, NUnit will launch, load ClassLibrary1.nunit, load the referenced assembly (ClassLibrary1.dll) and display all of the tests. Great!

   If we set a breakpoint on the instantiation of Class1 in the CreateClass1 test and we run the test - lo and behold the debugger breaks on the breakpoint.

   

   The power of this is not so much that you can debug your tests, but rather that you can easily debug any arbitrary part of your class library just by running the associated test. Joy!

   But we are no there just yet.

   If we were to check this into version control and someone else were to check it out, it would not work.

10. Making it version control friendly
    The reason it will not work is that, by default, the Debugging Configuration Properties (where we told it to launch NUnit) are stored in a user-specific file. In our case, ClassLibrary1.csproj.user. There is no point checking this file into version control because, when someone else checks out the project and loads it up, Visual Studio will create a new ClassLibrary1.csproj.user file for them - by copying values from the project file.

    So how do you check in the Debugging Configuration Properties?

    When I looked at the two files (ClassLibrary1.csproj and ClassLibrary1.csproj.user) I saw that the both had the same schema (VisualStudioProject | CSHARP | Build | Settings | Config). The relevant sections of the two files are as follows:

    Debug config for ClassLibrary1.csproj.user
    

    Debug config for ClassLibrary1.csproj
    

    So, I tried copying the Start-related attributes from the user file to the project file and…miracles of miracles…it works a treat.

    Debug config for ClassLibrary1.csproj after adding Start-related attributes
    

So, there we have it. All requirements met: we can use NUnit as the test harness, the debug harness and other developers can check out the project and do the same.

Note: If developers have installed NUnit in different drives or folders they can use the user-specific project file to overrride the default setting. They only need to do this once, either through the Configuration Property editor, or by editing the user-specific project file directly.