Unit testing seems to be one of those topics that will get people to arguing. What is a unit test? What value do they bring? How much time do they add to a project? You can’t write unit tests to test our code. And on and on… I think these questions continue to be asked because they are generally not answered well. To end this we need to find hard and fast answers to these questions. It is hard to argue for something if people can’t even agree on what it is.
u’nit test (noun) : An automated test, exercising a class, verifying that the class fulfills its responsibilities.
This is the absolute minimum needed for a definition of unit test. From here there could be many discussions about nice to haves with this definition (i.e. readability of the tests, naming of the test). Let’s examine this definition.
A unit test is not a unit test unless it is automated. If it is not automated then you must be using a debugger to step through a class to verify that it fulfills its responsibilities. This is a poor practice. It increases the viscosity of your environment; people will be less likely to perform this manual test every time it is needed. With automation thousands of unit tests can be run very quickly while thousands of manual tests will take a very long time. With out automation unit testing is next to worthless. The affect of unit tests is greater than their sum. To gain their full value all must be executed.
A unit test exercises just one class. What does this mean? Let’s say I have class Responder that needs to be tested. Responder has little public interface to speak of. The real meat is accessed by sending Responder messages through the file system. Verifying that Responder meets its responsibilities is also performed by receiving messages through the file system. Responder does not read and write messages to the files system itself, it uses a class called Tunnel. The temptation is to write a test that creates a Responder and rights and reads files to complete the test. Two thirds of what could go wrong is in the tunnels. There is more than one class being exercised. With a little redesign we can add some abstraction in to allow us to exercise just Responder. Exercising more than one class is an integration test.
A unit test will verify the subject class fulfills its responsibilities. As with exercising just one class a unit test is responsible to verify the responsibilities of just the subject class. When you are tempted to assert that an obligation of a class other than the subject is satisfied you are probably exercising more than the subject class. There should be unit tests for those other classes where their responsibilities are tested.
There are other levels of testing (integration, system…) where interactions are tested. Unit tests are mainly about testing the responsibilities of a class.
Naturally people feel exposed if the only tests on a system have tested the individual parts (other levels of testing should be exercised). What value is there in testing each part separately? It eliminates some of what you have to suspect when there is a bug. Unit tests have an affect on the code they test. The obvious affect is more reliable code. Some of the less obvious affects are loose coupling and ability to deal with change. This translates to the stakeholder as the project is more responsive to there needs as they figure them out. We all know that what is asked for at the beginning of the project and what is asked for by the end of the project can be very different things. Unit testing provides a means to balance these opposing forces: refactoring. It facilitates stepped, fine grained, or low level refactoring. With out true unit tests refactoing is a tricky business. In fact if your customer is asking for a change and you have good unit test coverage of your product the tests can be used to illustrate the risk involved with the change (i.e. before the change is applied X of Z tests are passing, after the change is applied Y of Z are passing).
If you are diligent there are other benefits to unit tests as well. Unit Tests can be, when written well, used as developer documentation. They can serve as executable design artifacts (similar to FiT tests). As a coach or architect they can be invaluable for identifying tangles, clutter, and bad or missing design.
When I here we can’t write unit tests for our code I think you don’t know how to use the tools of OOP. I have yet to come across a design problem that can not be solved with OOP to allow for simple and easy testing. It gets difficult when the subject code can not or will not be changed. Most times in these cases there is no simple easy way to create unit tests. Depending on the situation change may be necessary to create unit tests. Many times it is simply taking the time to do the right thing (i.e. creating an Object Mother). I think something that would go a long way to answering this question cold would be a set of patterns for common testing problems.
I think the most difficult question I have encountered is “How much time do they add to a project?”. I don’t think this question can be answered well in a general way. I think that for it to be affective it must address the specifics of the project(s) the questioner is thinking of. There are many variables and possible responses to the affects that unit testing will have on a project. Some developers get it right away, others take longer to understand how to practice TDD. It will reduce the amount of time spent debugging, and debugging is a laborious task. It will reduce the amount of bugs and recursion reaching the test team. The rapid feedback given to the developers should reduce the time needed to produce a feature or fix a bug. They no longer need to wait for the test team to perform verification to gain confidence that they have completed the task (and with the added bonus of not breaking anything else). The short simple (and some would say lame) answer is “Over the course of the project they will not add time but reduce the time needed”. The trick in the answer is “over the course of the project”. I would respond to that answer with “When will I first see the time savings?”. I am not sure I can win this line of questioning yet. It will invariably head to asking for a quantifiable or measurable difference: something tangible. The only way to kill this one is to find a way to make this question moot.
One of the interesting things about this last question is how it tempts me (and would guess others) to bring in other agile practices to the conversation. I am not sure if this is a good or a bad thing. There is defiantly a strong relation to other agile practices: refactoring, Continuous Integration, Continuous Design… At first I feel that their inclusion will bolster the case for unit tests and provide insight into their value. By including them in the discussion their value now seems to depend much on these other practices.
|W|P|112709950662818320|W|P|Understanding Unit Testing|W|P|
I think there is a parallel between what FiT is doing for requirements and what unit tests can do for Continuous Design. Specifically FiT provides executable requirements and unit tests provide executable design artifacts. Grant-it you must write your unit tests to be communicative. Those migrating from BDUF often feel insecure without design artifacts. There are many other things that make those migrating from BDUF feel insecure but I think this one can be solved by overloading the purpose of unit tests. Viewing unit tests as design artifacts can be a specially helpful at a shop where pair programming is not practiced. Design reviews are the counter to pair programming in the non-agile world, and unit tests could be used to review the proposed design. Having the design expressed as executable tests instills much more confidence in me than design artifacts like documents and diagrams. I can have greater assurance that the design I am reviewing will actually come to be. As well I am assured that the design artifact will stay in sync with the product. So there is an extra step between red and green when not practicing pair programming, design review by examination of the unit tests. Something to be careful of in these design reviews: reviewers are likely to think ahead to the future. The reviewers need to adhere to "Do the Simplest Thing that Could Possibly Work" and "You Aren't Going to Need It". Where interaction between entities larger than classes need to be reviewed FiT tests and fixtures can be included in the review. They should probably be included in the review anyway. If a shop can make this practice work it should save time. Instead of producing both tests (unit and FiT) and design docs they need only create the tests. The quality of the altered process should be the same if not better. The previous process being: create design artifacts, design review, red, green. Notice the lack of refactor? That is because there was a design specified up front, and the goal is to implement that design. The goal of the design was to adhere to "Do the Simplest Thing that Could Possibly Work" and "You Aren't Going to Need It” while expressing the new feature. Refactoring is done in the design phase.|W|P|112705348775346482|W|P|Continuous Design: Were did design go?|W|P|
So I have always thought that BDUF was flawed. I never enjoyed it, something just felt wrong. Over the last year I have been participating in agile experimentation at my work. For most new features we would have to document a design (UML and text) and hold a design review before proceeding to create unit tests. What was the purpose of these design reviews? To make sure that the implementor was going to express the values held by the architect in the code. It’s not that the developer was not trusted to fulfill the requirement, they were not trusted to hold and express the values of the architect (In this case the values are of OOP). So this indicates to me that the design documentation and review are to direct the implementor as to how they will fulfill the requirement.
I was very attached to this project and its success. I would monitor the CCNET server and review most every change made to the product. When I saw that the design was going astray I would talk with the author of the disruptive change to see what could be done to right the design. In monitoring the product in this fashion I found that design documentation and review were rarely translated into the expected code. This says to me that no amount of direction can get a developer that does not understand and or hold the desired values to produce code that adheres to them.
So I am sure you are asking how does this relate to Continuous Design. Well I don’t have a solution to how to get good code out of developers in opposition to the desired values. It feels like to me that Continuous Design has something to offer here. You might be saying use pair programming. Well that just seems like one developer (the one with the understanding) is doing all the work and the other developer (the one lacking understanding) is getting trained (if they are open to it). That doesn’t really sound like pair programming to me (maybe I don’t understand pair programming well?). At any rate I like Continuous Design better that BDUF. I think it is more productive. It reminds me of the Ron Jeffries quote:
"If I've got six months to build a system, then I'll spend six months building it. I'll also spend six months designing it, and another six months testing it. The good news is that it's the same six months".
So I have recently moved. Before the move I had a ten minute commute, now I have an hour and fifteen minute commute. To fill the time with something besides music I began listening to .NET Rocks. Listening to two .NET Rocks episodes a day is a bit much. I need a little more variety.
I have a list of papers, articles, and books that I have being meaning to read forever. It was almost the perfect time to get to them, the commute that is, except that I had to keep the car on the road. What I needed was someone to read them to me, like my laptop. So I went looking for a text to speech engine. I found plenty and chose AT&T’s PleaseRead. I liked it best for its record to MP3 feature. Now I have two and a half hours a day to listen to all that reading material that has been piling up.|W|P|112603526418952142|W|P|Listen When You Can't Read|W|P|