Yesterday I heart loads of excuses why the tests have not been written. I immedietaly remembered Misko Hevery’s session at amazing geecon conference (be sure to attend next year!). Misko told a story about influencing developers to write more tests, testable code and TDD. I liked his slide where he collected all the excuses he heart from devs explaining why they haven’t written tests. Basically, Misko’s conclusion was that the only valid reason of not writing tests/TDD was lack of skill. TDD or ability to write tests is a skill like any other and it is no shame if you haven’t acquired it yet!
However, some of the excuses I heart yesterday were beyond any common sense:
- this is not really worth testing
- this is a small application
- we don’t have time
- (my favorite one) tests would make it harder to refactor
That was all bullshit and the only reason the tests were not written was that it was hard to write them. Devs didn’t know how to write tests for the DAO layer (how to ensure db is in correct state, how to maintain test database, etc.). Devs didn’t know how to write integration tests because of lack of experience with tools like Selenium. So they didn’t write any tests, not even plain jUnits.
Half a year ago, the same development team wrote loads of tests achieving high quality code base & high test coverage. Those days they wrote a django application. Django is a modern framework with built-in means of testing. It significantly contributed to the fact that developers wrote tests without cheating themselves with it’s-not-worth-testing kind of excuses (complete list above).
The interesting thing about the situation I encounter yesterday was this: junior developer wanted to write tests for some DAO code that did not have any tests (actually, there were zero tests at all…). So I asked senior developer responsible for architecture to help out setting up a TestCase that can prepare connection to test database, etc. Instead of helping, the senior guy started his mantra of excuses trying to convince junior developer not to write tests. Fortunately, I was there…
May 12, 2009 at 1:14 pm |
I sympathize with the “tests make it harder to refactor” excuse. Over-reliance on mocking makes production code hard to change because the test failures become difficult to update and understand. Common techniques for testing legacy code (package-local methods, reflection) make production code hard to refactor because implementation changes break tests. However, your conclusion is still relevant. When someone says “tests make it harder to refactor” what they really mean is “I don’t have the skill to write tests that help instead of hinder refactoring”. Especially with legacy code, you can’t always refactor to a testable design (especially safely!), and if you don’t have a testable design then you’re stuck mocking/hacking poor tests together.
Michael Feathers talks about “creating islands of testability” in legacy code. That is great advice to follow. But, that also means there are oceans of code with no tests and low value in testing. So when we say we “create islands of testability” we are implicitly affirming the fact that some code is not worth testing and testing that code properly would take too much time.
Argh. It’s hard not to get angry about the reams of legacy code I have to deal with… but it’s a reality.
May 12, 2009 at 1:26 pm |
Spot on!
I guess I could imagine sympathizing with all those excuses on the occasion. It’s just they are too often just mere excuses made up to silence the conscience of a man who doesn’t know how to go about testing some feature or legacy piece. The important thing is to realize that it’s truly a valid excuse and no shame at all!
May 13, 2009 at 9:55 am |
Hi Guys, I’m a project manager working with business-critical and safety-critical systems, so apologies in advance that my technical competence is no longer what it used to be.
One of the critical factors for these high-reliability systems is that code must be fully testable and maintainable. Consequently all the code is run through a McCabe complexity analyser. If the McCabe value is too large it means the code cannot be readily tested or maintained and needs to be restructured. The project rule is that developers must check the McCabe value of their code which cannot go into Config Control or the build if it is too high.
Similarly with legacy code, it’s worthwhile running McCabe against the existing code and having a look at those items with a large complexity count. Experience has shown that the most complex items generate most of the problems. Also, when opening a code item for maintenance then run McCabe to see if the complexity count can be reduced to a more acceptable level.
McCabe complexity has mostly been used on applications written in C/C++ and Ada, but McCabe analysers are now available for Java although I don’t know if they cover DAO.
The acceptable McCabe threshold is up to the projects coding standards but for rough guidance; embedded safety-critical code typically has a McCabe of 10 or less, business critical IT a McCabe limit of 20 or 30 or less. For an old nightmare legacy system (written in C & VB) I initially set the McCabe threshold to 40 and this gave a good steer in identifying the really problematic code items; we found one code item with a McCabe of over 130!
Whatever the implementation language the principle of reducing complexity still applies, and the effort to check the McCabe at the developer level is small in comparison to the overal impact untestable or untested code has on the overal project costs and timescales.
Apologies if this is not relevant or fairly obvious.
Regards
May 13, 2009 at 10:07 am |
Don’t worry – you’ve actually got a very good point – the metric. If only the project had a metric exposing the problem of missing tests – probably a coverage tool rather than McCabe complexity tracker.
My digression on McCabe: couple of years ago we’ve set up pmd to fail the build if complexity was over 8
It worked OK for geographically divided team that did not have any formal code inspection in place. Recently, I’d rather choose pair-programming or more-less formal review process instead of rigid static analysis in the build.