Make Your Own Unit Test Mutant Army

Why are mutants even needed?

Well, in order for us to answer that question we are to determine what is a test and why do we need those in the first place. Why’s easy. Nowadays proper unit test writing skills are even more important that Java skills themselves for unexplainable (well, explainable, yet unreasonable) reasons thus making testing services enter a new stage. But if we are talking seriously, then unit tests are necessary because

  1.  They are the thing verifying the fact that code works
  2. They are the skeleton for future regression testing

But how may you be certain you are writing just the right tests? Sure you can measure all with code coverage (meaning how much code was tested) but will it be showing the right picture? Sure you may cover an impressive share of code (some companies are even requiring 70-90+%) because this seems as quite a proof the code works well. But will tests that are written (to speak the hard truth) to simply please management will have meaning? Are they testing the right things? Test coverage has nothing to do with test quality as it is not doing the most important thing any test has to do. It’s not verifying the outcome.

That is when mutants are to enter the stage

You may get some help in this field from something as unexpected as mutants. Or Mutation Testing in other words. Mutation Testing is the process of creation of something very much alike your mutant army, meaning thousands of ‘codebase mutants’. What is a mutant? A very small singular change in the codebase (like changing one sign, etc.)

Example: Notice the differences in the code highlighted with red.

// Before:
2 if(amount > THRESHOLD) {
3   // .. do something ..
4 }
6 // After:
7 if(amount >= THRESHOLD) {
8   // .. do something ..
9 }

This is basically one of many possible mutants and unit tests are to be run for every single one of those fellows. Which may lead us to:

  • Dead mutants. A failing test is the best possible outcome actually. That means your mutant is out-cold and you are now sure that your tests are doing a good job asserting your code. That is why one should use Mutation Testing. If we are being general it verifies the fact that tests are asserting the code. The knowledge you could not have gained by mere code coverage.
  • Live mutants. This happens when none of the tests have failed. That is the bad thing as it means that the changes into the logic we have made for this operation was not verified by the test. That means the test was not too much accurate and if somebody had made the very same mistake, only by accident, your tests would have been OK with it. Scary, right?

But, as we all know, nothing in this world is without a price and such a practice has some pitfalls as well. And performance will issue number one here. There are reasons to why many things were not implemented in the past. Imagine 80-es CPU melting just because unit tests. And now imagine adding something else. But, with today’s computers this shouldn’t be too much of a problem.


TestFort Blog

About TestFort Blog

TestFort blog is an official blog of TestFort QA Lab company and is dedicated to various QA and software testing issues.

no comments yet

Be the first to comment this post!

Would you like to share your thoughts?

Software testing & QA blog by TestFort © 2017