When Testing Meets Code Review: Why and How Developers Review Tests

Davide Spadini, Maurício Aniche, Margaret-Anne Storey, Magiel Bruntink, Alberto Bacchelli

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

33 Citations (Scopus)
341 Downloads (Pure)

Abstract

Automated testing is considered an essential process for ensuring software quality. However, writing and maintaining high-quality test code is challenging and frequently considered of secondary importance. For production code, many open source and industrial software projects employ code review, a well-established software quality practice, but the question remains whether and how code review is also used for ensuring the quality of test code. The aim of this research is to answer this question and to increase our understanding of what developers think and do when it comes to reviewing test code. We conducted both quantitative and qualitative methods to analyze more than 300,000 code reviews, and interviewed 12 developers about how they review test files. This work resulted in an overview of current code reviewing practices, a set of identified obstacles limiting the review of test code, and a set of issues that developers would like to see improved in code review tools. The study reveals that reviewing test files is very different from reviewing production files, and that the navigation within the review itself is one of the main issues developers currently face. Based on our findings, we propose a series of recommendations and suggestions for the design of tools and future research.
Original languageEnglish
Title of host publicationProceedings of the 40th International Conference on Software Engineering, ICSE '18
Place of PublicationNew York, NY
PublisherAssociation for Computing Machinery (ACM)
Pages677-687
Number of pages11
ISBN (Electronic)978-1-4503-5638-1
DOIs
Publication statusPublished - 2018
EventICSE 2018: 40th International Conference on Software Engineering - Gothenburg, Sweden
Duration: 27 May 20183 Jun 2018
Conference number: 40
https://www.icse2018.org/

Conference

ConferenceICSE 2018
Country/TerritorySweden
CityGothenburg
Period27/05/183/06/18
Internet address

Bibliographical note

Accepted Author Manuscript

Keywords

  • software testing
  • automated testing
  • code review
  • Gerrit

Fingerprint

Dive into the research topics of 'When Testing Meets Code Review: Why and How Developers Review Tests'. Together they form a unique fingerprint.

Cite this