What are different mutation testing methods?

Mutation testing is another methodology of software testing. It can be defined as the software testing methodology which involves modification of source code or programs or the byte code of the program in a way that is particularly small but significant.

Mutation is known by two other names:
- program mutation
- mutation analysis.

SOME IMPORTANT FACTS
- The mutated code is tested using various cases and the test suite that doesn’t detects and rejects the code that has been mutated is usually considered to be defective in nature.

- The required mutations are carried out on the basis of some well defined operators known as mutation operators.

- These mutation operators are well known to mimic typical programming bugs or errors and some times they also force the creation of tests which are significantly valuable tests.

- Best example of such test is given by the test which derives each expression to the value zero.

- The purpose of the mutation testing is basically to allow the tester or software developer to develop effective test cases so that the weaknesses or the vulnerabilities of the software system or application can be located or detected in the test data that is being used as the input for the program or in the sections of the program code.

- These sections are chosen as such that they are rarely used and accessed during the execution of the software system or the application.

- The mutation testing is carried out to ensure or verify that the implemented software program or application is correct or not.

Still the creation of the mutation testing poses some doubts.
- The doubt is whether the mutation tests are correct or not?
- Whether they sufficiently cover all the specifications and the requirements that have lead to the origination of the implementation of the test cases?

Mutation testing represents a technological problem of “who will guard the guards?”
- In the 19s, the mutation testing was pioneered to detect or locate the weaknesses and the potential vulnerabilities of the test suites.

- A theory was put forward regarding the behavior of the mutation testing.

- It stated that if a mutation was done without affecting the behavior of the program or output, then it meant that either the code that had been mutated did not execute or the employed testing suite wasn’t able to detect the injected mutation.

- In order to make one mutation work, one has to introduce many mutations all over the program.

- This in turn leads to the compilation of a large number of copies of the program code.

- The increase in the use of unit testing frame works and object oriented programming languages has led to the increase in the creation new mutation testing tools for other programming languages.

METHODS FOR MUTATION TESTING
- Mutation testing is carried out by selecting few mutation operators and then applying them to the source code using one at a time for each correct piece of the program code.

- The resultant of application of a mutation operator is known as mutant.

- If the mutant is detected during the testing, then the mutant is said to be killed.
- Strong mutation testing or strong mutation coverage ensures that the test suite is effective in detecting the mutants.
- Weak mutation coverage or weak mutation testing can be related to code coverage methods.
- It takes less number of calculations to ensure the effectiveness of the weak mutation testing.
- In some cases equivalent mutants are also used.
- In those cases it becomes impossible for a test suite to kill such a mutant.

- Equivalent mutation proves to be the greatest obstacle in the path of practical usage of the mutants.
- Equivalent mutants require more efforts for testing.
- A variety of mutation operators have been discovered by the researchers.

Mutation Testing : how it is performed, benefits, operators and tools.

Mutation Testing is a powerful method for finding errors in software programs. Mutation testing involves deliberately altering a program’s code, then re-running a suite of valid unit tests against the mutated program. A good unit test will detect the change in the program and fail accordingly. Mutation testing is expensive to run, especially on very large applications. Mutation Testing is complicated and time-consuming to perform without an automated tool.

How Mutation testing is performed?

- Create a mutant software which is different from the original software by one mutation.
- Each of the mutant software has one fault.
- Test cases are applied to the original software and the mutant software.
- Results are evaluated. The test case that is applied is wrong if the mutant software as well as the original software produces the same result. The test case is right if the test case detects fault in the software.

Benefits of Mutation Testing

- Introduces a new level of error detection.
- Uncover errors in code that were previously thought impossible to detect automatically.
- The customer will receive a more reliable and bug free software.

On what factors Mutation testing depends ?

- It depends heavily on the types of faults that the mutation operators are designed to represent.
- Mutation operators means certain aspects of the programming techniques, the slightest change in which may cause the program to function incorrectly.

Mutation Operators and Tools

Some mutation operators for languages like Java, C++ etc. are :
- Changing the access modifiers, like public to private etc.
- Static modifier change.
- Argument order change.
- Super keyword deletion.
- Essay writing services

Tools like Jester, Pester, Nester and Insure++ are some of the tools that are available for mutation testing.

Testing Approaches: Functional Testing versus Structural Testing and Mutation testing

In functional testing, functionality of the module is tested and structure is not considered. Test cases based on specifications and internals of modules are not considered. This type of testing is also known as black box testing. The objective here is to ensure that `correct results'' are obtained when `good inputs'' are applied to the various parts of the design, and when `bad inputs'' are applied the design operates in a predictable manner. Functional testing can therefore be considered as concentrating on checking that the data paths operate correctly.

Structural testing is used to test the implementation of the program. Structural testing, also known as white box or open box testing, is normally applied to sequential HDL code and concentrates on checking that all executable statements within each module. Structural testing involves :
- Statement coverage : This ensures that each and every statement is tested. Software tools called profilers are used to carry this statement coverage.
- Branch coverage : Each and every condition is taken, and inputs are given in such a way that each branch is executed at least once.
- Path Coverage : To test loops, this is required. For example the statement
for(i=o;i<=100;i++) is executed 101 times.

Mutation testing is required to ensure that the software does not fail. It is a good debugging mechanism. The program is modified slightly to obtain mutants of the program. Different mutants are tested with the same test cases. If the mutant fails, and the actual program works correctly, confidence is gained in the program and test cases are considered good. To produce mutants, mutation operators are defined.
- Constant replacement
- Variable replacement
- Arithmetic operator replacement
- Relational operator replacement
- Goto label replacement