It seems that it takes more effort to improve coverage as you approach 100%. Is this a futile exercise in diminishing returns or is there real value in being at 100% coverage?
|
0
|
|
|
|
|
1
|
In true test-driven development no code is written without a failing test. So 100% coverage is the norm under those conditions. But "coverage" is a somewhat meaningless term; I'm sure we've all seen code that had high test coverage and yet still contained defects. So it's more important to find team behaviours that ensure the product's completeness and correctness compared to any given set of requirements. |
||
|
|
|
0
|
I worked with a development team about 2 years ago who had an interesting take on 100%. By getting to 100% coverage and maintaining at 100% coverage, they ensured that no new untested code was allowed on the project - this was a monumental breakthrough in unit testing best practices. Note that there are work arounds to avoid coverage lapses in unnecessary areas such as generated code which may not ever be used and therefore doesn't need testing (think generated proxies for talking to web services). |
||
|
|
|
0
|
For our product, slightly less than 90% is covered with unit tests. Our team has been using TDD for years and are very comfortable writing unit tests, but 100% coverage seems like an unrealistic goal. 90% seems more achievable. |
||
|
|
|
0
|
The coverage percentage which can be reached depends on the programming language and the tools used to measure the coverage. Reaching 100% of all non-dead code with disciplined use of TDD happens naturally. The problem is covering those lines which are required by the language syntax, but will never be executed. For example in Java, the private constructor of a utility class which is meant to be never instantiated, or a catch clause for a checked exception which will never be thrown (for example ByteArrayOutputStream.close() is declared to throw IOException even though the method's implementation is empty and even its javadoc says that it will not throw an exception). Also some code coverage tools may calculate the coverage incorrectly and give false positives (for example reporting abstract methods and interfaces as not covered). Also remember that 100% line coverage does not mean that all execution paths and all states have been tested. To really cover a class, the programmer needs to write tests for all interesting states in which an object can be and for all transitions to other states. AFAIK, there are no tools which can measure that. |
||
|
|
|
0
|
The language and tool set is a big factor here. If your tools count imports, declarations, etc., your coverage may be much lower than 80%--but still may test everything that's important. Statistics lie. What do you hope to get from 100% coverage? |
||
|
|
|
0
|
I think the target should be 100%, but it does not have to become an absolute number. There was a pretty good series of podcasts between Joel Spolsky and Bob Martin (see http://blog.objectmentor.com/articles/2009/02/06/on-open-letter-to-joel-spolsky-and-jeff-atwood). They both had very interesting arguments, and came to an agreement at the end. |
||
|
|
