Write a Blog >>
ICSE 2021
Mon 17 May - Sat 5 June 2021

Modern programming follows the continuous integration (CI) and continuous deployment (CD) approach rather than the traditional waterfall model. Even the development of modern programming languages uses the CI/CD approach to swiftly provide new language features and to adapt to new development environments. Unlike in the conventional approach, in the modern CI/CD approach, a language specification is no more the oracle of the language semantics because both the specification and its implementations (interpreters or compilers) can co-evolve. In this setting, both the specification and implementations may have bugs, and guaranteeing their correctness is non-trivial.

In this paper, we propose a novel N+1-version differential testing to resolve the problem. Unlike the traditional differential testing, our approach consists of three steps: 1) to automatically synthesize programs guided by the syntax and semantics from a given language specification, 2) to generate conformance tests by injecting assertions to the synthesized programs to check their final program states, 3) to detect bugs in the specification and implementations via executing the conformance tests on multiple implementations, and 4) to localize bugs on the specification using statistical information. We actualize our approach for the JavaScript programming language via JEST, which performs N+1-version differential testing for modern JavaScript engines and ECMAScript, the language specification describing the syntax and semantics of JavaScript in a natural language. We evaluated JEST with four JavaScript engines that support all modern JavaScript language features and the latest version of ECMAScript (ES11, 2020). JEST automatically synthesized 1,700 programs that covered 97.78% of syntax and 87.70% of semantics from ES11. Using the assertion-injected JavaScript programs, it detected 44 engine bugs in four different engines and 27 specification bugs in ES11.

Fri 28 May

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

15:05 - 16:05
4.3.1. Analyzing System Properties: Correctness, Determinism, RealizabilityTechnical Track at Blended Sessions Room 1 +12h
Chair(s): Maria Teresa Baldassarre Department of Computer Science, University of Bari
15:05
20m
Paper
JEST: N+1-version Differential Testing of Both JavaScript Engines and SpecificationACM SIGSOFT Distinguished PaperArtifact ReusableTechnical TrackArtifact Available
Technical Track
Pre-print Media Attached
15:25
20m
Paper
Unrealizable Cores for Reactive Systems SpecificationsArtifact ReusableTechnical Track
Technical Track
Shahar Maoz Tel Aviv University, Israel, Rafi Shalom Tel Aviv University, Israel
DOI Pre-print Media Attached
15:45
20m
Paper
Verifying Determinism in Sequential ProgramsArtifact ReusableTechnical Track
Technical Track
Rashmi Mudduluru University of Washington, Jason Waataja UW CSE, Suzanne Millstein University of Washington, Michael D. Ernst UW CSE
Pre-print Media Attached

Sat 29 May

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change