Looking for more information on how to do PHP the right way? Check out PHP: The Right Way

Dave Marshall:
Probing Test Suite Quality with Mutation Testing
Jan 08, 2015 @ 12:09:42

In this recent post to his site Dave Marshall looks at a method for evaluating the overall quality of your suite of unit tests with help from mutation testing.

100% code coverage should never really be a goal. [...] I feel pursuing 100% coverage in a PHP project is a particularly poor idea as our tooling generally only provides Line Coverage. [...] There are more reasonable coverage metrics to use to measure the quality of a test suite. Sebastian Bergmann and Derick Rethans are working hard on bringing some of these options to us, but for now we're limited to line coverage.

He talks about the difference between line, branch and condition coverage types (with code examples) and which allows for more effective and quality tests to be written. He then talks about the results of an experiment to achieve 100% coverage on the Router component in the Aura project. He found the problem using mutation testing - changing values in the production code to make sure the tests break. He also links over to a new mutation testing tool that's been released to help with this kind of thing, humbug, and some of the results it can report.

Mutation testing is a great thing to have a grasp of in theory, but it's not particularly easy to practice. The tools are very hard to write and then their output is often hard to understand or interpret effectively. I wouldn't recommend practicing with mutation testing on a regular basis, but it's certainly worth considering on the odd occasion.
tagged: test suite quality mutation testing unittest coverage

Link: http://davedevelopment.co.uk/2015/01/07/probing-test-suite-quality-with-mutation-testing.html