Friday, April 19, 2013

Automated software testing is a major investment

Passing test suites attest as to the quality of the code being tested. Just as importantly, it gives the developer confidence to evolve the code, i.e. to refactor or add new features without being afraid that the changes break existing functionality. Without the safety net provided by the test suite, evolving software can be a perilous and nerve wrecking activity to which few developers survive intact.

In a nutshell, a good test suite is critical to the success of a project. You might think that since test code does not directly impact end-users, it should be cheaper to write and maintain. On the contrary, if one is serious about automated testing, then test suites do not come cheap.

For example, in the logback project, the size of test code roughly matches the size of the code being tested. Thus, one might estimate the cost of test code to be equivalent to that of the code being tested. In my experience, this is unfortunately not true. It turns out that reliable test code is surprisingly hard to write, especially for code with many external dependencies. So, in reality we spend more time on the test code. This is a price we are willing to pay in order to guarantee the quality of our product.

Given the above, it seems that consultants insisting that their clients attain 100% code-coverage are misleading their flock or they have test writing skills that many developers, including the author, do not possess.

No comments: