[+] Wiki Tools

Wiki Relationships:

Admin Tools:

Upload file

Basics of Mutation Testing

By Manish Marathe

1 The Approach
2 Mutation Operators
3 Mutation Testing tools
4 References and related links

The Approach

Mutation testing is a fault-based testing technique that is based on the assumption that a program is well tested if all simple faults are predicted and removed; complex faults are coupled with simple faults and are thus detected by tests that detect simple faults.

The idea of mutation testing was introduced as an attempt to solve the problem of not being able to measure the accuracy of test suites. The thinking goes as follows: Let’s assume that we have a perfect test suite, one that covers all possible cases. Let’s also assume that we have a perfect program that passes this test suite. If we change the code of the program (this process is called mutation) and we run the mutated program (mutant) against the test suite, we will have two possible scenarios:

The results of the program were affected by the code change and the test suite detects it. If this happens, the mutant is called a killed mutant.
The results of the program are not changed and the test suite does not detect the mutation. The mutant is called an equivalent mutant.

The ratio of killed mutants to the total mutants created measures how sensitive the program is to the code changes and how accurate the test suite is.

The effectiveness of Mutation Testing depends heavily on the types of faults that the mutation operators are designed to represent. By mutation operators we mean certain aspects of the programming techniques, the slightest change in which may cause the program to function incorrectly. For example, a simple condition check in the following code may perform viciously because of a little change in the code.

Original Code: for (x==1) { ….}
Mutated Code: for (x<=1) { ….}

In the example above the ‘equality condition’ was considered as a mutation operator and a little change in the condition is brought into effect as shown in the mutated code and the program is tested for its functionality.

Edit

Mutation Operators

Following could be some of the Mutation Operators for Object-Oriented languages like Java, C++ etc.

Changing the access modifiers, like public to private etc.
static modifier change.
Argument order change.
super keyword deletion.

There are many more like these which can be studied from the second reference given below.

Edit

Mutation Testing tools

Following are some of the practically implemented Mutation Testing tools which are being used.

Jester: A Mutation Testing tool in Java (Open Source)
Pester: A Mutation Testing tool in Python (Open Source)
Nester: A Mutation Testing tool in C# (Open Source)
Insure++: A commercial testing tool offered by Parasoft, Inc, which leverages Mutation Testing.

Edit