Write a Blog >>
AST 2021
Fri 28 - Sat 29 May 2021
co-located with ICSE 2021
Thu 20 May 2021 15:30 - 15:45 at AST Room - Mutation Testing and Debugging Chair(s): Francisco Gortázar

Mutation testing is used to evaluate the effectiveness of test suites. In recent years, a promising variation called extreme mutation testing emerged that is computationally less expensive. It identifies methods where their functionality can be entirely removed, and the test suite would not notice it, despite having coverage. These methods are called pseudo-tested. In this paper, we compare the execution and analysis times for traditional and extreme mutation testing and discuss what they mean in practice. We look at how extreme mutation testing impacts current software development practices and discuss open challenges that need to be addressed to foster industry adoption. For that, we conducted an industrial case study consisting of running traditional and extreme mutation testing in a large software project from the semiconductor industry that is covered by a test suite of more than 11,000 unit tests. In addition to that, we did a qualitative analysis of 25 pseudo-tested methods and interviewed two experienced developers to see how they write unit tests and gathered opinions on how useful the findings of extreme mutation testing are. Our results include execution times, scores, numbers of executed tests and mutators, reasons why methods are pseudo-tested, and an interview summary. We conclude that the shorter execution and analysis times are well noticeable in practice and show that extreme mutation testing supplements writing unit tests in conjunction with code coverage tools. We propose that pseudo-tested code should be highlighted in code coverage reports and that extreme mutation testing should be performed when writing unit tests rather than having a separate session to analyze the findings. For that, future research should investigate how to perform extreme mutation testing while writing unit tests such that the results are available fast enough but still meaningful.

Thu 20 May

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

15:00 - 16:15
Mutation Testing and DebuggingAST 2021 at AST Room
Chair(s): Francisco Gortázar Universidad Rey Juan Carlos
15:00
30m
Long-paper
SuMo: A Mutation Strategy for Solidity Smart Contracts
AST 2021
Pre-print Media Attached
15:30
15m
Short-paper
Extreme mutation testing in practice: An industrial case study
AST 2021
Maik Betka University of Stuttgart, Stefan Wagner University of Stuttgart
Pre-print Media Attached
15:45
30m
Long-paper
Extending Hierarchical Delta Debugging with Hoisting
AST 2021
Daniel Vince , Renáta Hodován University of Szeged, Hungary, Daniella Bársony University of Szeged, Department of Software Engineering, Ákos Kiss University of Szeged, Hungary
Pre-print Media Attached

Information for Participants
Thu 20 May 2021 15:00 - 16:15 at AST Room - Mutation Testing and Debugging Chair(s): Francisco Gortázar
Info for room AST Room:

Go directly to this room on Clowdr