Friday 7 May 2010

Unit testing - it's about the feedback cycle

When I start introducing unit testing to someone that is not familiar with it, one of the first complaints I hear is that it takes more time to develop code and unit tests as opposed to just code. This is not true and actually in most cases it takes less time and as a side effect you end up with a set of unit tests that will make your life easier in the future.
Let’s assume that every piece of code gets tested by its creator before it gets handed over to the QA guys. Pretty reasonable assumption, isn't it? If there are no unit tests the only way to test the functionality is to run the whole application or some kind of integration tests. This takes seconds if not minutes for each test case. You can run a well written unit test within milliseconds. You can run 100 unit tests within a couple of seconds(even with mstest as long as you use VS2010 RTM).
If a test takes minutes to execute it’s easy to loose focus and switch to something else for a while. We all know how expensive the context switching is. Unit tests give you instant feedback which helps you stay focused and more productive. Less time for a single test case means that you can test more cases which in turn leads to fewer bugs. Sure, once the code is unit tested you need to actually run it from within the application but this is more to make sure that all bits and pieces are correctly configured rather than to do extensive testing.
Now calculate what is the cost in terms of time of a bug found by your QA team. In such a case the following needs to happen:
  • a tester has to create a bug report
  • someone has to triage it
  • a developer needs to:
    • get familiar with the problem
    • recreate the problem
    • fix it
    • test it (without unit tests)
    • promote to the source control system
    • make sure the CI build is green
  • a tester needs to test the fix
In the best organized company I’ve seen, the total time of such an exercise would be around 2h assuming that the whole process finishes within a day or two since the initial checkin.
It’s all about the length of the feedback cycle. The shorter it is the better. If you find a bug with a unit test you loose a couple of minutes, if you let it through to QA environment or even worse to Production you loose hours or days(think about all the hours you spent with windbg :)).
If unit tests help you lower the number of bugs by 1 for a given feature then you end up with enough time to cover the task of writing them. Interesting, isn’t it?
There are many more advantages of unit testing that can even further reduce the time needed to write a piece of code. One of the most efficient ones is TDD which lets you drive the design of the code with unit tests. Crazy? No, it actually works very well. You can read more about it here.


  1. Hey Pawel, can you believe the place you used to work has come a long way: we aren't "code complete" until the feature has been coded and all corresponding unit tests have also been written and pass. In fact, the code base is unit testable by design now and is possible to do TDD - my favourite style of coding. I'm to present TDD to the team to encourage more devs to try it - it's awesome!