Mutation testing is becoming increasingly widely used to evaluate the quality of test suites, especially to test programs coded in widely used programming languages in the industry. Mutation tools have arisen to automate the technique in different languages, including C++. With the increasing use of this technique, new mutation operators modeling possible faults often emerge to improve its abilities and adapt the tools to new advanced features. In this work, mutation operators for the new C++ standards, defined in previous work, are implemented and applied to generate and execute mutants in real programs. With this study, the MuCPP mutation tool is updated with the inclusion of these new operators. In addition, the improvements suggested in the definition of those operators can be finally tested, and conclusions about their utility in practice can be drawn. The implemented operators are checked on a set of four C++ programs that use these advanced features. The results show significant differences with the previous manual analysis: the number of invalid mutants was reduced by 64%, and we found fewer alive mutants (88%) and an increase in dead mutants (31%). In summary, both the number of mutants incorrectly classified in the previous manual analysis and the number of mutants generated (particularly equivalent mutants) have been reduced.

Miguel Ángel Álvarez-García is a PhD student at the Department of Computer Science and Engineering at the University of Cadiz in Spain. He graduated in Computer Science Engineering from the University of Cadiz, Spain, in 2019, and he obtained a Master’s Degree in Systems and Computer Engineering Research also at the University of Cadiz, in 2020. He is a researcher of the UCASE Software Engineering Research Group. He works on projects related to Software Engineering focus on the Mutation Testing and Artificial Intelligence.