Comments on: excuses for not doing dev testing

By: szczepiq

szczepiq — Wed, 13 May 2009 10:07:26 +0000

Apologies if this is not relevant or fairly obvious.

Don’t worry – you’ve actually got a very good point – the metric. If only the project had a metric exposing the problem of missing tests – probably a coverage tool rather than McCabe complexity tracker.

My digression on McCabe: couple of years ago we’ve set up pmd to fail the build if complexity was over 8 It worked OK for geographically divided team that did not have any formal code inspection in place. Recently, I’d rather choose pair-programming or more-less formal review process instead of rigid static analysis in the build.

By: Dave Nicoll

Dave Nicoll — Wed, 13 May 2009 09:55:53 +0000

Hi Guys, I’m a project manager working with business-critical and safety-critical systems, so apologies in advance that my technical competence is no longer what it used to be.
One of the critical factors for these high-reliability systems is that code must be fully testable and maintainable. Consequently all the code is run through a McCabe complexity analyser. If the McCabe value is too large it means the code cannot be readily tested or maintained and needs to be restructured. The project rule is that developers must check the McCabe value of their code which cannot go into Config Control or the build if it is too high.

Similarly with legacy code, it’s worthwhile running McCabe against the existing code and having a look at those items with a large complexity count. Experience has shown that the most complex items generate most of the problems. Also, when opening a code item for maintenance then run McCabe to see if the complexity count can be reduced to a more acceptable level.
McCabe complexity has mostly been used on applications written in C/C++ and Ada, but McCabe analysers are now available for Java although I don’t know if they cover DAO.

The acceptable McCabe threshold is up to the projects coding standards but for rough guidance; embedded safety-critical code typically has a McCabe of 10 or less, business critical IT a McCabe limit of 20 or 30 or less. For an old nightmare legacy system (written in C & VB) I initially set the McCabe threshold to 40 and this gave a good steer in identifying the really problematic code items; we found one code item with a McCabe of over 130!
Whatever the implementation language the principle of reducing complexity still applies, and the effort to check the McCabe at the developer level is small in comparison to the overal impact untestable or untested code has on the overal project costs and timescales.

Apologies if this is not relevant or fairly obvious.
Regards

By: szczepiq

szczepiq — Tue, 12 May 2009 13:26:20 +0000

Spot on!

I guess I could imagine sympathizing with all those excuses on the occasion. It’s just they are too often just mere excuses made up to silence the conscience of a man who doesn’t know how to go about testing some feature or legacy piece. The important thing is to realize that it’s truly a valid excuse and no shame at all!

By: Hamlet D'Arcy

Hamlet D'Arcy — Tue, 12 May 2009 13:14:58 +0000

I sympathize with the “tests make it harder to refactor” excuse. Over-reliance on mocking makes production code hard to change because the test failures become difficult to update and understand. Common techniques for testing legacy code (package-local methods, reflection) make production code hard to refactor because implementation changes break tests. However, your conclusion is still relevant. When someone says “tests make it harder to refactor” what they really mean is “I don’t have the skill to write tests that help instead of hinder refactoring”. Especially with legacy code, you can’t always refactor to a testable design (especially safely!), and if you don’t have a testable design then you’re stuck mocking/hacking poor tests together.

Michael Feathers talks about “creating islands of testability” in legacy code. That is great advice to follow. But, that also means there are oceans of code with no tests and low value in testing. So when we say we “create islands of testability” we are implicitly affirming the fact that some code is not worth testing and testing that code properly would take too much time.

Argh. It’s hard not to get angry about the reams of legacy code I have to deal with… but it’s a reality.