Practical Definition of a Unit Test
The number of units in a “Unit Test” is typically one isolated software module. If there are two or more dependent software modules being tested, it is an integration test. Since a “software module” is often a class, a unit test class usually tests one class, and a unit test method usually tests one method. A true unit test does not contain volatile dependencies such as file I/O, networks, databases, web services, web servers, message queuing, randomness, or specific date/times.
The whole point of UnitTests is to reduce the scope of the system under test to a small subset that can be tested in isolation. Thus, by the ScientificMethod, we reduce the number of variables that may affect the results so we can derive useful information on whether the unit is actually correct.” — SunirShah
Also see http://artofunittesting.com/definition-of-a-unit-test/ for Roy Osherove’s definition of a good unit test.
Mocking Your Volatile Dependencies
One practical way to determine if a test contains volatile dependencies is to ask yourself, “Will this test run quickly and dependably on a bare-bones build server*?” If it cannot, then it’s because of a volatile dependency.
* The “build server” in question would be a minimalist install with just the .Net framework, possibly Visual Studio (although most of that wouldn’t be needed) and the required testing, mocking, and build libraries, but not any other runtimes, and certainly NOT IIS or SQL Server, at least not for builds.
Use Dependency Injection techniques to replace volatile dependencies with Test Doubles. Since most classes contain instances of other classes, you must design your classes so that volatile dependencies can be substituted for unit testing. If you don’t design your classes to use dependency injection you may only be able to write integration tests for them:
- Your unit tests will be limited to only very simple classes or have superficial coverage–they will often be of low value.
- While your integration tests may still be useful, you may find some of them to be brittle or impractical to automate and/or maintain.
If you have worked with Inversion of Control (IoC) techniques and/or DI Containers you’ll already be familiar with Dependency Injection. Understanding IoC and DI techniques are prerequisites to understanding design patterns and unit testing.
Don’t Reinvent the Wheel
When implementing Test Doubles, prefer a mocking framework over do-it-yourself Stubs. If you don’t do this you may end up with just as much or more test code than the code you are testing. A clean, simple, yet powerful mocking framework for .Net is NSubstitute (http://nsubstitute.github.io/)
NSubstitute is even simpler to use than Moq; Rhino Mocks was once popular but now suffers from having to support too much backwards compatibility; Microsoft Fakes comes with Visual Studio 2012 but it has not yet caught on.
“Good” Unit Tests
According to Roy Osherove’s “The Art of Unit Testing,” automated tests that take too long to run on a build system or require some external setup are probably not unit tests. A good unit should have these qualities:
- Automated piece of code that invokes a different method and then checks some assumptions on the logical behavior of that method or class.
- Written using a unit-testing framework.
- Written easily.
- Runs quickly.
- Can be executed repeatedly by anyone on the development team. It should run at the push of a button and/or on a build system.
A system that contains many components that are difficult or impossible to unit test often indicates high coupling with low cohesion. Low quality design is often difficult to test. Following good design practices like S.O.L.I.D principles leads to a more testable system.
One of the first things to watch out for are classes that directly or indirectly involve databases, files, and logging. Unit tests should be able to run in memory. Even if you have integration tests, it’s still a good idea to eliminate these volatilities from those classes if possible. When persistence is involved it introduces these problems:
- Time: usually orders of magnitude longer than memory access and subject to physical volatility of the I/O device.
- Location: behavior may differ from one environment to another.
- Also subject to physical volatility of the I/O device.
- Possibility of transient or intermittent sharing or locking issues.
- Hidden complexities:
- Rather than testing just the code logic you may find yourself working around the persistence mechanism itself (especially databases).
- If a network is involved it becomes yet another volatile factor.
Databases are difficult to test.
- If you are using a Repository object model (as recommended for MVC), you may be able to mock the repository.
- Some continuous integration environments allow for automated testing of a database. Although it is not true unit testing and requires a lot of setting up (tear down/recreate for testing), it may be a practical progression once you get a handle on your unit tests.
In .Net, if you need to test a non-public method (and have decided that it’s not a good idea to make the method public just for testing), use the InternalsVisibleTo assembly attribute in the Software Under Test’s [SUT’s] AssemblyInfo.cs file.The SUT has to know the name of the assembly that is testing it, so it is technically “backwards coupling” by name; however,
- it won’t break the SUT if the test assembly in the attribute goes away or if the name is incorrect, and
- in version 2.0 of the .Net framework the attribute was specifically added by Microsoft developers as a workaround to having to use reflection tricks to test private or internal methods and members.
Integration Tests Have Their Place
Automated integration testing has its place. Just because a test isn’t a good unit test doesn’t mean it’s a bad test–it might be a great automated integration test. You need to know when you are writing an integration test rather than a unit test because calling an integration test a unit test can cause problems when automating the tests. The volatile parts may also change temporarily or permanently, which may cause your tests to either break or become invalid.
You shouldn’t use integration testing as an excuse for not writing unit tests. If you can write good unit tests your integration tests will improve as well. You should design new code to be unit testable, but also use integration tests to cover important scenarios.