Write a Blog >>
ICSE 2021
Mon 17 May - Sat 5 June 2021

Software analytics have empowered software organisations to support a wide range of improved decision-making and policy-making. However, such predictions made by software analytics to date have not been explained and justified. Specifically, current defect prediction models still fail to explain why models make such a prediction and fail to uphold the privacy laws in terms of the requirement to explain any decision made by an algorithm. In this paper, we empirically evaluate three model-agnostic techniques, i.e., two state-of-the-art Local Interpretability Model-agnostic Explanations technique (LIME) and BreakDown techniques, and our improvement of LIME with Hyper Parameter Optimisation (LIME-HPO). Through a case study of 32 highly-curated defect datasets that span across 9 open-source software systems, we conclude that (1) model-agnostic techniques are needed to explain individual predictions of defect models; (2) instance explanations generated by model-agnostic techniques are mostly overlapping (but not exactly the same) with the global explanation of defect models and reliable when they are re-generated; (3) model-agnostic techniques take less than a minute to generate instance explanations; and (4) more than half of the practitioners perceive that the contrastive explanations are necessary and useful to understand the predictions of defect models. Since the implementation of the studied model-agnostic techniques is available in both Python and R, we recommend model-agnostic techniques be used in the future.

Thu 27 May

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

11:50 - 13:05
3.2.3. Defect Prediction: Bug Characterization & AnalysisJournal-First Papers / NIER - New Ideas and Emerging Results at Blended Sessions Room 3 +12h
Chair(s): Robert Feldt Chalmers | University of Gothenburg, Blekinge Institute of Technology
Watch out for Extrinsic Bugs! A Case Study of their Impact in Just-In-Time Bug Prediction Models on the OpenStack projectJournal-First
Journal-First Papers
Gema Rodríguez-Pérez University of Waterloo, Mei Nagappan University of Waterloo, Gregorio Robles Universidad Rey Juan Carlos
DOI Pre-print Media Attached
An Empirical Study of Model-Agnostic Techniques for Defect Prediction ModelsJournal-First
Journal-First Papers
Jirayus Jiarpakdee Monash University, Australia, Kla Tantithamthavorn Monash University, Hoa Khanh Dam University of Wollongong, John Grundy Monash University
Link to publication Pre-print Media Attached
On the cost and profit of software defect predictionJournal-First
Journal-First Papers
Steffen Herbold University of Göttingen
Link to publication DOI Pre-print Media Attached
Software Ticks Need No SpecificationsNIER
NIER - New Ideas and Emerging Results
Christoph Reichenbach Lund University
Pre-print Media Attached