What helped, and what did not? An Evaluation of the Strategies to Improve Continuous IntegrationTechnical Track
Wed 26 May 2021 05:00 - 05:20 at Blended Sessions Room 2 - 1.4.2. Continuous Integration
Continuous integration (CI) is a widely used practice in modern software engineering. Unfortunately, it is also an expensive practice — Google and Mozilla estimate their CI systems in millions of dollars. There are a number of techniques and tools designed to or having the potential to save the cost of CI or expand its benefit - reducing time to feedback. However, their benefits in some dimensions may also result in drawbacks in others. They may also be beneficial in other scenarios where they are not designed to help. In this paper, we perform the first exhaustive comparison of techniques to improve CI, evaluating 14 variants of 10 techniques using selection and prioritization strategies on build and test granularity. We evaluate their strengths and weaknesses with 11 different cost and time-to- feedback saving metrics on 100 real-world projects. We analyze the results of all techniques to understand the design decisions that helped different dimensions of benefit. We also synthesized those results to lay out a series of recommendations for the development of future research techniques to advance this area.
Tue 25 MayDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
16:40 - 17:35 | 1.4.2. Continuous IntegrationJournal-First Papers / Technical Track / NIER - New Ideas and Emerging Results at Blended Sessions Room 2 +12h Chair(s): Daniela Damian University of Victoria | ||
16:40 20mPaper | A Machine Learning Approach to Improve the Detection of CI Skip CommitsJournal-First Journal-First Papers Rabe Abdalkareem Queens University, Kingston, Canada, Suhaib Mujahid Concordia University, Emad Shihab Concordia University Link to publication DOI Pre-print Media Attached | ||
17:00 20mPaper | What helped, and what did not? An Evaluation of the Strategies to Improve Continuous IntegrationTechnical Track Technical Track Pre-print Media Attached | ||
17:20 15mPaper | ADEPT: A Socio-Technical Theory of Continuous IntegrationNIER NIER - New Ideas and Emerging Results Omar Elazhary University of Victoria, Margaret-Anne Storey University of Victoria, Neil Ernst University of Victoria, Elise Paradis University of Toronto Pre-print Media Attached |
Wed 26 MayDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
04:40 - 05:35 | 1.4.2. Continuous IntegrationTechnical Track / NIER - New Ideas and Emerging Results / Journal-First Papers at Blended Sessions Room 2 | ||
04:40 20mPaper | A Machine Learning Approach to Improve the Detection of CI Skip CommitsJournal-First Journal-First Papers Rabe Abdalkareem Queens University, Kingston, Canada, Suhaib Mujahid Concordia University, Emad Shihab Concordia University Link to publication DOI Pre-print Media Attached | ||
05:00 20mPaper | What helped, and what did not? An Evaluation of the Strategies to Improve Continuous IntegrationTechnical Track Technical Track Pre-print Media Attached | ||
05:20 15mPaper | ADEPT: A Socio-Technical Theory of Continuous IntegrationNIER NIER - New Ideas and Emerging Results Omar Elazhary University of Victoria, Margaret-Anne Storey University of Victoria, Neil Ernst University of Victoria, Elise Paradis University of Toronto Pre-print Media Attached |