BoKSS 2021 People: Armin Beer, Michael Felderer, Tobias Lorey, Stefan Mohacsi

… Testing is a core software development activity that has huge potential to make software development more sustainable. In this paper, we discuss how … of test planning, design, execution, and evaluation. …

AST 2021 When: Fri 21 May 2021 15:15 - 15:45 People: Cristiano Politowski, Fabio Petrillo, Yann-Gaël Guéhéneuc

… . This quality could be assessed and ensured through testing. However, to the best of our knowledge, little is known about video-game testing. In this paper, we want to understand how game developers perform game testing. We investigate, through …

AST 2021 When: Fri 21 May 2021 16:30 - 16:45 People: Maximiliano Agustin Mascheroni, Emanuel Agustin Irrazábal, Gustavo Rossi

… to testing at using practices like Continuous Delivery or Continuous Development. Thus, we propose Continuous Testing Improvement Model (CTIM) as a solution to the testing problems in continuous software development environments. It brings together …

MET 2021 When: Wed 2 Jun 2021 13:00 - 13:20 People: Zenghui Zhou, Zheng Zheng, Tsong Yueh Chen, Jinyi Zhou, Kun Qiu

… Recently, metamorphic testing (MT) has been used to augment test datasets by inclusion of new (follow-up) test cases transformed from existing (source) test … test dataset usually has higher fault detection capabilities. Intuitively …

AST 2021 When: Fri 21 May 2021 12:00 - 12:30 People: Leonardo Mariani, Mauro Pezze, Valerio Terragni, Daniele Zuddas

… Automatic generators of GUI tests often fail to generate semantically relevant test cases, and thus miss important test scenarios. To address this issue, test … meaningful GUI tests from test cases of applications with similar functionalities …

MET 2021 When: Wed 2 Jun 2021 18:00 - 18:30 People: Melvin Wolf, Jürgen Schönwälder

… described in this paper explores how metamorphic testing, a testing methodology alleviating the oracle problem, can be used to test the PALISADE library. Some faults are discovered and the applicability of metamorphic testing …

MET 2021 When: Wed 2 Jun 2021 12:30 - 13:00 People: Zhirui Zhang, Dave Towey, Zhihao Ying, Yifan Zhang, Zhi Quan (George) Zhou

… , which is the first step in penetration testing. They are tools that can help … Metamorphic Testing to test two scanners, Nmap and Masscan. Seven Metamorphic Relations … is reported. The results show the potential for Metamorphic Testing to evaluate …

AST 2021 When: Thu 20 May 2021 15:30 - 15:45 People: Maik Betka, Stefan Wagner

… Mutation testing is used to evaluate the effectiveness of test suites. In recent years, a promising variation called extreme mutation testing emerged … can be entirely removed, and the test suite would not notice it, despite having …

MET 2021 When: Wed 2 Jun 2021 16:40 - 17:00 People: M S Raunak, Megan Olsen

… Metamorphic testing has been shown to be useful in testing ``non-testable … when compared to standard software, we show that metamorphic testing …. In this paper we expand on our five years of prior work on metamorphic testing …

AST 2021 When: Thu 20 May 2021 12:30 - 13:00 People: Ali Sedaghatbaf, Mahshid Helali Moghadam, Mehrdad Saadatmand

… Generating test cases that can reveal performance issues in large and complex …. On the other hand, we have a limited test budget to execute test cases. What makes … details of these systems.In this paper, we present an automated test case …

deeptest2021 When: Tue 1 Jun 2021 11:20 - 11:30 People: Agatino Giuliano Mirabella, Alberto Martin-Lopez, Sergio Segura, Luis Valencia-Cabrera, Antonio Ruiz-Cortés

… Automated test case generation for RESTful web APIs is a thriving research … follow a black-box approach, where test cases are randomly derived from the API …, when testing real-world services, most random test cases tend to be invalid …

BotSE 2021 When: Fri 4 Jun 2021 19:00 - 19:15 People: Jordi Cabot, Lola Burgueño, Robert Clarisó, Gwendal Daniel, Jorge Perianez Pascual, Roberto Rodriguez-Echeverria

… on the company revenue or image. Unfortunately, the field of testing NLP-intensive bots is still in its infancy. This paper aims to characterize the testing properties … and evaluate future bot testing research initiatives. …

AST 2021 When: Thu 20 May 2021 12:00 - 12:30 People: Stefan Karlsson, Adnan Causevic, Daniel Sundmark

… . However, there is currently a lack of methods to automatically generate tests … producing GraphQL queries to test GraphQL APIs. This is achieved using a property … of the system under test. Our evaluation on a real world software system shows …

AST 2021 When: Fri 21 May 2021 12:30 - 12:45 People: Franz Wotawa, Ledio Jahaj, Lorenz Klampfl

… assurance and especially testing of vision systems in general. In this paper, we contribute to the area of testing vision software, and present a framework for the automated generation of tests for systems based on vision and image recognition …

Technical Track When: Thu 27 May 2021 12:45 - 13:05Fri 28 May 2021 00:45 - 01:05 People: Goran Petrović, Marko Ivanković, Gordon Fraser, René Just

… Various proxy metrics for test quality have been defined in order to guide developers when writing tests. Code coverage is particularly well established in practice, even though the question of how coverage relates to test quality …

EnCyCriS 2021 When: Thu 3 Jun 2021 09:00 - 09:30 People: Zelalem Mihret, Lingjun Liu

… The formal verification technique of model-checking can be used to derive test … representation notations. In this paper, we present a focused test case generation … by specific and putative attack behaviours. To this end, we devised test …

SoHeal 2021 When: Sat 29 May 2021 17:00 - 17:15 People: Fabiano Pecorelli, Fabio Palomba, Andrea De Lucia

… studies showed that test-related factors (e.g., code coverage) can predict the future … provided compelling evidence on the relation between tests and postrelease defects, yet they considered different test-related factors separately …

MET 2021 When: Wed 2 Jun 2021 18:30 - 18:50 People: Dilip Jagadeeshwarswamy Hiremath, Martin Claus, Wilhelm Hasselbring, Willi Rath

… Metamorphic testing seeks to validate software in the absence of test oracles. Our application domain is ocean system modeling, where test oracles rarely … test scenarios using machine learning. We extended our previously proposed method …

deeptest2021 When: Tue 1 Jun 2021 13:00 - 14:00 People: Mark Harman, Natalija Gucevska

… interactions. This talk presents an overview of Facebook’s WW system, its testing (using automated regression, end-to-end and metamorphic testing), and exciting open problems and applications of Software Testing for Smart Simulation systems built …

AST 2021 When: Fri 21 May 2021 15:45 - 16:15 People: Lori Flynn, William Snavely, Zachary Kurtz

… propose using static analysis test suites (i.e., repositories of benchmark programs that are purpose-built to test static analysis tools) as a novel source … various static analyzers on the Juliet test suite, and we automatically derived …

MET 2021 When: Wed 2 Jun 2021 10:40 - 11:00 People: Muhammad Iqbal, Jia Cheng Han, Zhi Quan (George) Zhou, Dave Towey

… . To ensure their safety, simulation testing is essential. In this research, we … ADAS testing standards: We tested the Lane Keeping Assist System (LKAS), which is a prebuilt ADAS module of MATLAB and Simulink. We first tested LKAS using …

CloudIntelligence 2021 When: Sat 29 May 2021 17:07 - 17:20 People: Nutcha  Temiyasathit, Tao Yang, Karan Luthra, Nick Ruff, Petar Zuljevic, Ethan Benowitz, Boris Baracaldo, Oytun Eskiyenenturk, Xin Fu

… Most A/B testing frameworks in the industry are based on randomized allocation of users to control and test groups. Such systems are not suitable to test …), a framework containing hypothesis testing library and visualization tool. SEAT …

AST 2021 When: Fri 21 May 2021 15:00 - 15:15 People: Chidera Biringa, Gokhan Kul

… Although there are many automated software testing suites, they usually focus on unit, system and interface testing. However, especially software updates … experience. In this paper, we propose a novel automated user experience testing …

Technical Papers When: Mon 17 May 2021 10:04 - 10:08 People: Diego Marcilio, Carlo A. Furia

… Exceptions often signal faulty or undesired behavior; hence, high-quality test … study of exceptional tests—which exercise exceptional behavior—in 1,157 open-source Java projects hosted on GitHub.

We analyzed JUnit exceptional tests …

MET 2021 When: Wed 2 Jun 2021 17:00 - 17:30 People: Madeline Diep, Rohan Reddy Mekala, Ujjwal Ayyangar, Gudjon Magnusson, Adam Porter, Wolfgang Losert, Mikael Lindvall

… . Metamorphic testing has been shown to be a powerful technique for scientific data analysis applications. Our idea is to integrate metamorphic testing into the BRAIN … where we applied metamorphic testing to one step in the calcium imaging …

AST 2021 When: Fri 21 May 2021 17:15 - 17:45 People: Aitor Arrieta, Jon Ayerdi, Miren Illarramendi, Aitor Agirre, Goiuria Sagardui, Maite Arratibel

… validate this software, test oracles are necessary. A typical approach in industry is to use regression oracles. These oracles have to execute the test input both, in the software version under test and in a previous software version …

MET 2021 When: Wed 2 Jun 2021 12:00 - 12:30 People: Hyejin Park, Taaha Waseem, Wen Qi Teo, Ying Hwei Low, Mei Kuan Lim, Chun Yong Chong

… adopt Metamorphic Testing technique to evaluate the robustness of the model … StackGAN-v2 algorithm and test the pre-trained model provided by the original authors … relation, from which test cases are generated. Further, metamorphic relationships …

Research When: Fri 21 May 2021 02:10 - 02:20 People: Anthony Peruma, Emily Hu, Jiajun Chen, Eman Abdullah AlOmar, Mohamed Wiem Mkaouer, Christian D. Newman

… It is good practice to name test methods such that they are comprehensible … automated support for writing or maintaining the names of test methods. This can lead to inconsistent and low-quality test names and increase the maintenance cost …

Technical Papers When: Tue 18 May 2021 17:11 - 17:15 People: Golnaz Gharachorlu, Nick Sumner

… Given a failing test case, test case reduction yields a smaller test case … and error with smaller test cases. Current techniques speed up reduction …-guided approach to speed up test case reduction. The approach trains a model …

Registered Reports When: Mon 17 May 2021 10:19 - 10:22 People: Sarra Habchi, Maxime Cordy, Mike Papadakis, Yves Le Traon

… Background: Test flakiness is identified as a major issue that compromises the regression testing process of complex software systems. Flaky tests manifest … their trust in test suites. Both industrial reports and research studies highlighted …

Technical Track When: Thu 27 May 2021 15:45 - 16:05Fri 28 May 2021 03:45 - 04:05 People: Dong Jae Kim, Nikolaos Tsantalis, Tse-Hsun (Peter) Chen, Jinqiu Yang

… Since the introduction of annotations in Java 5, the majority of testing … design. This adoption affected the testing practices in every step of the test lifecycle from fixture setup and test execution to fixture teardown. Despite …

Mining Challenge When: Mon 17 May 2021 17:18 - 17:21 People: Anthony Peruma, Christian D. Newman

… of unit testing. Through unit tests, developers verify the correctness of production source code, thereby verifying the system’s intended behavior under test. However, unit test code is subject to issues, ranging from bugs in the code to poor …

Research Papers When: Thu 20 May 2021 17:14 - 17:16 People: Eduard Paul Enoiu, Robert Feldt

… Automated testing tools typically create test cases that are different from … tests harder to understand, and thus results in tools providing less support … cognitive processes of testers. The framework helps map test design steps and criteria …

DS - Doctoral Symposium When: Fri 21 May 2021 18:25 - 18:55 People: Olivia Rodriguez Valdes

… …

SEIP - Software Engineering in Practice When: Wed 26 May 2021 18:50 - 19:10Thu 27 May 2021 06:50 - 07:10 People: Mark Harman, John Ahlgren, Maria Eugenia Berezin, Elena Dulskyte, Inna Dvortsova, Johann George, Natalija Gucevska, Erik Meijer, Justin Spahr-Summers, Kinga Bojarczuk, Silvia Sapora, Maria Lomeli

… ). MIA is used to test Facebook’s Web Enabled Simulation, built on a web … of test flakiness and the unknowable oracle problem. It uses metamorphic testing to automate continuous integration and regression test execution. MIA also plays …

AST 2021 When: Fri 21 May 2021 16:45 - 17:15 People: Alessio Bucaioni, Fabio Di Silvestro, Inderjeet Singh, Mehrdad Saadatmand, Henry Muccini, Thorvaldur Jochumsson

… In this work, we report on our experience in defining and applying a model-based approach for the automatic generation of test scripts for product variants … testing artefacts and making their generation automatic. We demonstrate …

SRC - ACM Student Research Competition When: Tue 25 May 2021 11:30 - 13:30 People: David Berend

… to the traditional software development process, testing the DL software to uncover its …. Although recent progress has been made in designing novel testing techniques for DL software, the distribution of generated test data is not taken into consideration …

Technical Track When: Wed 26 May 2021 16:45 - 17:05Thu 27 May 2021 04:45 - 05:05 People: Vasudev Vikram, Rohan Padhye, Koushik Sen

… for automatically generating a corpus of concise test inputs for programs … and the corresponding problem of generating meaningful test cases to provide to students. The prior state-of-the-art solution is a combination of fuzzing and test-case …

NIER - New Ideas and Emerging Results When: Thu 27 May 2021 12:10 - 12:25Fri 28 May 2021 00:10 - 00:25 People: Annibale Panichella, Cynthia C. S. Liem

… Mutation testing is a well-established technique for assessing a test suite’s …, mutation testing has been extended to machine learning (ML) systems, and deep … extent currently used mutation testing techniques in DL are actually in line …

Technical Track When: Thu 27 May 2021 15:45 - 16:05Fri 28 May 2021 03:45 - 04:05 People: Alan Romano, Zihe Song, Sampath Grandhi, Wei Yang, Weihang Wang

… Flaky tests have gained attention from the research community in recent years and with good reason. These tests lead to wasted time and resources and reduce the reliability of the test suites and build systems they affect. However, most …

JSEET - Joint Track on Software Engineering Education and Training When: Thu 27 May 2021 12:30 - 12:50Fri 28 May 2021 00:30 - 00:50 People: Pedro Delgado-Pérez, Inmaculada Medina-Bulo, Miguel Ángel Álvarez-García, Kevin Jesús Valle-Gómez

… Testing is a crucial activity in the development of software systems … graduates with adequate testing skills and preparation in this field. A challenge in software testing education is to make students perceive the benefits …

Technical Track When: Fri 28 May 2021 11:30 - 11:50Fri 28 May 2021 23:30 - 23:50 People: Melina Vidoni

… Testing Technical Debt (TTD) occurs due to shortcuts (non-optimal decisions) taken about testing; it is the test dimension of technical debt. R is a package …, datasets, tests, documentation and examples. R’s package structure makes …

JSEET - Joint Track on Software Engineering Education and Training When: Fri 28 May 2021 15:25 - 15:45Sat 29 May 2021 03:25 - 03:45 People: Lex Bijlsma, Niels Doorn, Harrie Passier, Harold Pootjes, Sylvia Stuurman

… an introductionary course on programming without any formal education on testing, we asked … to write test cases to decide whether a specific method is correct … that students do not test systematically, while most of them think they do test …

Technical Track When: Thu 27 May 2021 15:25 - 15:45Fri 28 May 2021 03:25 - 03:45 People: Abdulrahman Alshammari, Christopher Morris, Michael Hilton, Jonathan Bell

… When developers make changes to their code, they typically run regression tests to detect if their recent changes (re)introduce any bugs. However, many tests … apparent cause. Flaky tests are a significant nuisance in the development process …

SRC - ACM Student Research Competition When: Tue 25 May 2021 19:00 - 21:00 People: Yufeng Chen

… . As software systems mature, the cost of running their entire regression test suite can become significant. Selective Regression Testing (SRT) is a technique that executes only a subset of tests the regression test suite can detect software …

Technical Papers When: Mon 17 May 2021 10:15 - 10:19 People: Guillaume Haben, Sarra Habchi, Mike Papadakis, Maxime Cordy, Yves Le Traon

… Industrial reports indicate that flaky tests are one of the primary concerns of software testing mainly due to the false signals they provide. To deal … at (automatically) identifying flaky tests with encouraging results. However, to reach …

Technical Track When: Thu 27 May 2021 17:10 - 17:30Fri 28 May 2021 05:10 - 05:30 People: Swaroopa Dola, Matthew B Dwyer, Mary Lou Soffa

… testing of the safety and trustworthiness of these systems. In the last few years, there have been a number of research efforts focused on testing DNNs. However the test generation techniques proposed so far lack a check to determine …

SRC - ACM Student Research Competition When: Tue 25 May 2021 11:30 - 13:30 People: Pablo Valle

… . However, in order to deploy autonomous vehicles, effective testing techniques … testing such systems. However, as in other complex systems, determining the outcome of a test in the driving quality on an AV can be extremely complex. To solve …

SRC - ACM Student Research Competition When: Tue 25 May 2021 11:30 - 13:30 People: Agustín Mista

… We present MUTAGEN, a fully automated mutation-oriented framework for property-based testing. Our tool uses novel heuristics to improve the performance of the testing loop, and it is capable of finding complex bugs within seconds. We …

Technical Track When: Wed 26 May 2021 11:40 - 12:00Wed 26 - Thu 27 May 2021 People: Jingyi Wang, Jialuo Chen, Youcheng Sun, Xingjun Ma, Dongxia Wang, Jun Sun, Peng Cheng

… . One popular direction is deep learning testing, where adversarial examples … with the help of certain testing metrics. However, recent studies have revealed that the commonly used neuron coverage metrics by existing DL testing approaches …

Technical Papers When: Tue 18 May 2021 17:01 - 17:05 People: Andre Hora

… Test coverage is largely used to assess test effectiveness. In practice … that will not be executed during tests is irrelevant and can actually harm the analysis … study to understand code exclusion practices in test coverage. We mine 55 Python …

Q-SE 2021 When: Wed 2 Jun 2021 15:15 - 15:45 People: José Campos, André Souto

… …

Research Papers When: Fri 21 May 2021 03:09 - 03:11 People: Luiz Fernando Capretz, Ali Bounassif, Harous Saad

… …

WAIN'21 When: Mon 31 May 2021 09:00 - 09:20 People: Qunying Song, Emelie Engstrom, Per Runeson

… …

Replications and Negative Results (RENE) When: Thu 20 May 2021 16:20 - 16:30 People: Bruno Henrique Pachulski Camara, Marco Aurélio Graciotto Silva, André T. Endo, Silvia Regina Vergilio

… …

Technical Track When: Tue 25 May 2021 11:10 - 11:30Tue 25 May 2021 23:10 - 23:30 People: Shengcheng Yu, Chunrong Fang, Zhenfei Cao, Xu Wang, Tongyu Li, Zhenyu Chen

… Crowdsourced testing is increasingly dominant in mobile application (app) testing, but it is a great burden for app developers to inspect the incredible number of test reports. Many researches have been proposed to deal with test reports …

Social and Networking When: Tue 25 May 2021 11:30 - 12:00Tue 25 - Wed 26 May 2021

… …

Technical Track When: Thu 27 May 2021 12:25 - 12:45Fri 28 May 2021 00:25 - 00:45 People: Wei Ma, Thierry Titcheu Chekam, Mike Papadakis, Mark Harman

… In many cases developers would like to test the program changes they introduce. To achieve this, a mutation testing formulation capable of capturing the altered program behaviours that are not adequately tested, is needed …

SRC - ACM Student Research Competition When: Tue 25 May 2021 19:00 - 21:00 People: Miguel Ángel Álvarez-García

… Mutation testing is becoming increasingly widely used to evaluate the quality of test suites, especially to test programs coded in widely used programming … tested, and conclusions about their utility in practice can be drawn …

Journal-First Papers When: Wed 26 May 2021 19:30 - 19:50Thu 27 May 2021 07:30 - 07:50 People: Josie Holmes, Iftekhar Ahmed, Caius Brindescu, Rahul Gopinath, He Zhang, Alex Groce

… extremely useful for automated test generation. It is both highly language-dependent … for automated testing. In this article, we make use of a heuristic based on LOC counts for tested functions to dramatically improve the effectiveness of automated …

Technical Track When: Fri 28 May 2021 11:50 - 12:10Fri 28 - Sat 29 May 2021 People: Shengcheng Yu, Chunrong Fang, Yexiao Yun, Yang Feng

… ), like WeChat. In such a situation, recording and replaying test scripts is one of the most popular automated mobile app testing approaches. However, such approach … test scripts for multiple platforms aimed at completely the same test …

Technical Track When: Tue 25 May 2021 12:25 - 12:45Wed 26 May 2021 00:25 - 00:45 People: Zan Wang, Hanmo You, Junjie Chen, Yingyi Zhang, Xuyuan Dong, Wenbin Zhang

… Deep Neural Network (DNN) testing is one of the most widely-used ways to guarantee the quality of DNNs. However, labeling test inputs to check the correctness … of DNN testing, even the whole process of DNN development. To relieve the labeling …

Technical Track When: Tue 25 May 2021 12:45 - 13:05Wed 26 May 2021 00:45 - 01:05 People: Giovani Guizzo, Justyna Petke, Federica Sarro, Mark Harman

… Improvement (SE for AI). We conjecture that existing Regression Test Selection … of RTS in future test-based automated software improvement work. Finally, we …

Technical Track When: Wed 26 May 2021 19:10 - 19:30Thu 27 May 2021 07:10 - 07:30 People: Song Wang, Nishtha Shrestha, Abarna Kucheri Subburaman, Junjie Wang, Moshi Wei, Nachiappan Nagappan

… Automatic unit test generation that explores the input space and produces effective test cases for given programs have been studied for decades. Many unit test generation tools that can help generate unit test cases with high structural …

NIER - New Ideas and Emerging Results When: Thu 27 May 2021 19:40 - 19:55Fri 28 May 2021 07:40 - 07:55 People: Viet-Man Le, Alexander Felfernig, Mathias Uta, David Benavides, Jose Galindo, Trang Tran

… to the automated testing and debugging of variability models. The algorithm helps …

Journal-First Papers When: Wed 26 May 2021 18:50 - 19:10Thu 27 May 2021 06:50 - 07:10 People: Ezekiel Soremekun, Esteban Pavese, Nikolas Havrikov, Lars Grunske, Andreas Zeller

… When a program has been tested on some sample input(s), what additional input does one test next? To further test the program, one needs to construct inputs … samples.
This paper presents a novel test generation approach that employs …

Technical Track When: Wed 26 May 2021 14:30 - 14:50Thu 27 May 2021 02:30 - 02:50 People: YAN ZHENG, Yi Liu, Xiaofei Xie, Yepang Liu, Lei Ma, Jianye Hao, Yang Liu

… Web testing has long been recognized as a notoriously difficult task.Even nowadays, web testing still mainly relies on manual efforts in many cases while automated web testing is still far from achieving human-level performance. Key …

NIER - New Ideas and Emerging Results When: Thu 27 May 2021 19:20 - 19:35Fri 28 May 2021 07:20 - 07:35 People: Luca Guglielmo, Andrea Riboni, Giovanni Denaro

… …

Journal-First Papers When: Thu 27 May 2021 16:30 - 16:50Fri 28 May 2021 04:30 - 04:50 People: Wei Ma, Mike Papadakis, Anestis Tsakmalis, Maxime Cordy, Yves Le Traon

… …

Technical Track When: Fri 28 May 2021 11:50 - 12:10Fri 28 - Sat 29 May 2021 People: Thodoris Sotiropoulos, Stefanos Chaliasos, Vaggelis Atlidakis, Dimitris Mitropoulos, Diomidis Spinellis

… for systematically testing Object-Relational Mapping (ORM) systems. Our approach leverages differential testing to establish a test oracle for ORM-specific bugs … systems under test. To tackle the challenge that ORMs lack a common input language …

Technical Track When: Wed 26 May 2021 11:20 - 11:40Wed 26 May 2021 23:20 - 23:40 People: Weisi Luo, Xiaoyue Run, Dong Chai, Jiang Wang, Chunrong Fang, Zhenyu Chen

… to pay attention to their quality. Testing is one of the major methods of quality assurance. However, existing testing techniques focus on the quality of DL models … and libraries). Inspired by the success stories of fuzz testing, we design …

GI 2021 When: Sun 30 May 2021 22:30 - 22:40 People: Linsey Kitt, Myra Cohen

… In this paper we argue for using many partial test suites instead of one full test suite during program repair. This may provide a pool of simpler, yet correct …. To support this idea, we present some insight obtained running APR partial test …

Technical Track When: Tue 25 May 2021 12:45 - 13:05Wed 26 May 2021 00:45 - 01:05 People: Pinjia He, Clara Meister, Zhendong Su

… . To evaluate RTI, we use Purity to test Google Translate and Bing Microsoft …

SEIP - Software Engineering in Practice When: Fri 28 May 2021 20:10 - 20:30Sat 29 May 2021 08:10 - 08:30 People: Junjie Zhu, Teng Long, Atif Memon

… tests. We identify four specific challenges and address them by developing a new general methodology to automatically author and maintain tests. In particular, we … of test inputs and expected outputs; we use perturbation of the data to obtain …

Technical Track When: Thu 27 May 2021 15:25 - 15:45Fri 28 May 2021 03:25 - 03:45 People: Peilun Zhang, Yanjie Jiang, Anjiang Wei, Victoria Stodden, Darko Marinov, August Shi

… on 200 open-source Java projects and detect 275 tests that fail due to wrong … that can fix these violations in both the test code and the main code. We … can fix 119 tests from our sample of 275 tests. We have already reported 102 fixes …

AST 2021 When: Fri 21 May 2021 17:45 - 18:15 People: Haiyang Sun, Andrea Rosà, Daniele Bonetta, Walter Binder

… TESA, a new tool that automatically assembles a test suite for any package in the npm registry. The test suite includes 1) tests written for the target package and usually hosted in its development repository, and 2) tests selected from …

AST 2021 When: Thu 20 May 2021 13:00 - 13:15 People: Nicky Williams

… Branch coverage of source code is a very widely used test criterion. Moreover … of assertion violations, certain runtime errors and various other types of test objective. Indeed, establishing that a large number of test objectives …

SESoS/WDES 2021 When: Thu 3 Jun 2021 17:15 - 17:30 People: Francisco Ramirez, Carlos Mera-Gómez, Rami Bahsoon, Yuqun Zhang

… . This approach improves testing of applications, where modularization can limit … and testing and we have categorised the concerns. Our results indicate that (i) missing …

MET 2021 When: Wed 2 Jun 2021 18:50 - 19:20 People: Andrei Lascu, Matt Windsor, Alastair F. Donaldson, Tobias Grosser, John Wickerson

… Metamorphic testing requires the availability of a suitable set of metamorphic relations (MRs) for the application domain of interest. A software testing practitioner interested in using metamorphic testing is thus blocked unless they can …

Technical Track When: Fri 28 May 2021 15:05 - 15:25Sat 29 May 2021 03:05 - 03:25 People: Jihyeok Park, Seungmin An, Dongjun Youn, Gyeongwon Kim, Sukyoung Ryu

… +1-version differential testing_ to resolve the problem. Unlike the traditional differential testing, our approach consists of three steps: 1) to automatically … specification, 2) to generate conformance tests by injecting assertions to the synthesized …

Journal-First Papers When: Thu 27 May 2021 21:10 - 21:30Fri 28 May 2021 09:10 - 09:30 People: Jinfu Chen, Weiyi Shang, Emad Shihab

… propose an approach that automatically predicts whether a test would manifest … that are associated with each test. For each commit, we build random forest classifiers that are trained from all prior commits to predict in this commit whether each test …

AST 2021 When: Thu 20 May 2021 15:45 - 16:15 People: Daniel Vince, Renáta Hodován, Daniella Bársony, Ákos Kiss

… Minimizing failing test cases is an important preprocessing step on the path of debugging. If much of a test case that triggered a bug does not contribute … considerably. However, test case reduction itself can be a time consuming task, especially …

AST 2021 When: Thu 20 May 2021 15:00 - 15:30 People: Morena Barboni, Andrea Morichetta, Andrea Polini

… must be thoroughly tested before their release. Testing is one of the main …. Mutation testing is a powerful approach for assessing the fault-detection capability of a test suite. In this paper, we propose SuMo, a novel mutation testing tool …

SRC - ACM Student Research Competition When: Tue 25 May 2021 11:30 - 13:30 People: Jinyang Shao

… to employ white-box testing on DNNs since the output of a single neuron is inexplicable. Existing work conducted metamorphic testing for object detection via image … propose an automatic approach to testing object detection via 3D reconstruction …

GI 2021 When: Sun 30 May 2021 22:40 - 22:50 People: Ryan Dougherty, Xi Jiang

… Testing a large-scale system requires understanding how each of the components interact with each other, which is the subject of interaction testing. Covering arrays are a well-studied object, but ``traditional'' representations …

deeptest2021 When: Tue 1 Jun 2021 11:00 - 11:20 People: Michael Weiss, Rwiddhi Chakraborty, Paolo Tonella

… criteria for Deep Learning (DL) testing. As an adequacy criterion, it has been used to assess the strength of DL test suites. In addition, it has also been used …, computation of the SA metric for a test suite can be prohibitively expensive …

deeptest2021 When: Tue 1 Jun 2021 14:00 - 14:20 People: Niranjhana Narayanan, Karthik Pattabiraman

… , and so ensuring the reliability of its operations is important. Mutation testing … on improving the quality of test data. We present an open source data mutation tool …

Technical Track When: Thu 27 May 2021 10:00 - 10:20Thu 27 May 2021 22:00 - 22:20 People: Sungjae Hwang, Sungho Lee, Jihoon Kim, Sukyoung Ryu

… a specification and generate test programs. JUSTGEN receives the JNI specification written … cases using an SMT solver. It then generates test programs that trigger the behaviors of unspecified cases. Using the generated tests, we empirically study …

NIER - New Ideas and Emerging Results When: Wed 26 May 2021 14:50 - 15:05Thu 27 May 2021 02:50 - 03:05 People: Mukelabai Mukelabai, Thorsten Berger, Paulo Borba

… …

AE - Artifact Evaluation People: Abdulrahman Alshammari, Christopher Morris, Michael Hilton, Jonathan Bell

… …

DEMO - Demonstrations When: Fri 28 May 2021 10:55 - 11:25 People: Aniya Aggarwal, Samiulla Shaikh, Sandeep Hans, Swastik Haldar, Rema Ananthanarayanan, Diptikalyan Saha

… …

MET 2021 When: Wed 2 Jun 2021 10:20 - 10:40 People: Yao Deng, Guannan Lou, Xi Zheng, Tianyi Zhang, Miryung Kim, Huai Liu, Chen Wang, Tsong Yueh Chen

… in autonomous driving. Recent work leverages metamorphic testing to improve defect … driven development-based metamorphic testing framework.BMT enables domain experts … to a corresponding metamorphic relation and synthesizes meaningful test inputs using …

WETSEB 2021 When: Mon 31 May 2021 10:30 - 10:50 People: Giacomo Ibba, Antonio Pierro, Marco Di Francesco

… . We tested several models, some of which returned an overall accuracy of 99pt …

WETSEB 2021 When: Mon 31 May 2021 15:40 - 16:00 People: Noama Samreen, Manar H. Alalfi

… smart contracts under test (SCUTs) to identify patterns that are potentially …

Technical Track When: Tue 25 May 2021 12:05 - 12:25Wed 26 May 2021 00:05 - 00:25 People: Linghan Meng, Yanhui Li, Lin Chen, Zhi Wang, Di Wu, Yuming Zhou, Baowen Xu

… models and select the more suitable ones w.r.t. the whole testing context. Due …

Technical Track When: Thu 27 May 2021 19:55 - 20:15Fri 28 May 2021 07:55 - 08:15 People: Mohammad Bajammal, Ali Mesbah

… be satisfied. While some accessibility testing tools exist, most only perform rudimentary …

GI 2021 When: Sun 30 May 2021 21:30 - 21:55 People: Ibrahim Mesecan, Michael C. Gerten, James I. Lathrop, Myra Cohen, Tomas Haddad Caldas

… programs must be validated. Researchers have recently designed a software testing … APR framework. We use a testing infrastructure built in the Matlab SimBiology …

RoSE 2021 When: Wed 2 Jun 2021 15:15 - 15:30 People: André Santos, Alcino Cunha, Nuno Macedo

… Verification, and Property-based Testing. It has been applied to multiple real …

RoSE 2021 When: Wed 2 Jun 2021 14:15 - 14:30 People: Alexandre Albore, David Doose, Christophe Grand, Charles Lesire-Cabaniols, Augustin Manecy

… skill models. This skill management layer helps to structure and test …

FormaliSE 2021 When: Tue 18 May 2021 14:30 - 15:00 People: Alexander Knüppel, Leon Schaer, Ina Schaefer

… implementation or as test oracles during automatic test case generation. As writing strong …