On the feasibility and challenges of synthesizing executable Espresso tests
Several tools have been proposed to automatically test Android applications, achieving outstanding results in terms of both code coverage and crash discovery. While useful for crash reproduction and bug-fixing, these tools usually do not present the generated interactions in a format that motivates developers to read and modify such tests later on. This hinders the ability of developers to add those tests to their existing test suites, or adapt them to new scenarios – common practices in modern software development where tests are maintained and evolve alongside production code.
In this work we present an empirical study on the challenges of automatically synthesizing Espresso test suites from sequences of interactions over widgets. We build on top of the MATE testing tool and implement a prototype that enables this study. The prototype is then evaluated on 12 open-source Android apps, followed by an analysis and discussion of challenges and limitations. We also include feedback from developers of open-source projects and an industrial app.
Our empirical study shows that the creation of Espresso tests is difficult, mostly due to the lack of unique properties to unambiguously identify specific widgets in the UI. This problem is aggravated in some cases by the incomplete or ambiguous definition of GUI components and layouts. It also points out that further research is needed to find ways to improve the testability of Android apps either manually or automatically. Nonetheless, the feedback received indicates that the synthesized Espresso tests are still useful for projects with few or no test cases, serving as a starting point for creating new ones.
Wed 18 MayDisplayed time zone: Eastern Time (US & Canada) change
09:05 - 10:25
Session 4: Test generation IIAST 2022 at AST room
Chair(s): Maurizio Leotta DIBRIS, University of Genova, Italy
|Testing non-testable programs using association rules|
Antonia Bertolino CNR-ISTI, Emilio Cruciani Gran Sasso Science Institute, L'Aquila, Italy, Breno Miranda Federal University of Pernambuco, Roberto Verdecchia Vrije Universiteit Amsterdam
|On the feasibility and challenges of synthesizing executable Espresso tests|
Iván Arcuschin Moreno University of Buenos Aires, Argentina, Christian Ciccaroni , Juan Pablo Galeotti University of Buenos Aires, José Miguel Rojas The University of SheffieldDOI Pre-print
|CrawLabel: Computing Natural-Language Labels for UI Test Cases|
Yu Liu The University of Texas at Austin, Rahulkrishna Yandrapally University of British Columbia, Canada, Anup Kalia , Saurabh Sinha IBM Research, Rachel Tzoref-Brill IBM Research, Ali Mesbah University of British Columbia (UBC)