Java Unit Testing Tool Competition - Eighth Round

Xavier Devroey; Sebastiano Panichella; Alessio Gambi

doi:10.1145/3387940.3392265

Java Unit Testing Tool Competition - Eighth Round

Xavier Devroey, Sebastiano Panichella, Alessio Gambi

Software Engineering

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

16 Citations (Scopus)

84 Downloads (Pure)

Abstract

We report on the results of the eighth edition of the Java unit testing tool competition. This year, two tools, EvoSuite and Randoop, were executed on a benchmark with (i) new classes under test, selected from open-source software projects, and (ii) the set of classes from one project considered in the previous edition. We relied on an updated infrastructure for the execution of the different tools and the subsequent coverage and mutation analysis based on Docker containers. We considered two different time budgets for test case generation: one an three minutes. This paper describes our method- ology and statistical analysis of the results, presents the results achieved by the contestant tools and highlights the challenges we faced during the competition.

Original language	English
Title of host publication	Proceedings - 2020 IEEE/ACM 42nd International Conference on Software Engineering Workshops, ICSEW 2020
Publisher	ACM DL
Pages	545-548
Number of pages	4
ISBN (Electronic)	9781450379632
DOIs	https://doi.org/10.1145/3387940.3392265
Publication status	Published - 2020
Event	42nd International Conference on Software Engineering: ICSE 2020 - Seoul, Korea, Republic of Duration: 27 Jun 2020 → 19 Jul 2020

Conference

Conference	42nd International Conference on Software Engineering
Country/Territory	Korea, Republic of
City	Seoul
Period	27/06/20 → 19/07/20
Other	Virtual/online event due to COVID-19 online presentations

Bibliographical note

Virtual/online event due to COVID-19

Keywords

tool competition
benchmark
software testing
test case generation
unit testing
Java
JUnit

Access to Document

10.1145/3387940.3392265

tool-competition-2020Accepted author manuscript, 416 KB

Cite this

@inproceedings{2057bad9650a4f2cac571093574fa83e,

title = "Java Unit Testing Tool Competition - Eighth Round",

abstract = "We report on the results of the eighth edition of the Java unit testing tool competition. This year, two tools, EvoSuite and Randoop, were executed on a benchmark with (i) new classes under test, selected from open-source software projects, and (ii) the set of classes from one project considered in the previous edition. We relied on an updated infrastructure for the execution of the different tools and the subsequent coverage and mutation analysis based on Docker containers. We considered two different time budgets for test case generation: one an three minutes. This paper describes our method- ology and statistical analysis of the results, presents the results achieved by the contestant tools and highlights the challenges we faced during the competition.",

keywords = "tool competition, benchmark, software testing, test case generation, unit testing, Java, JUnit",

author = "Xavier Devroey and Sebastiano Panichella and Alessio Gambi",

note = "Virtual/online event due to COVID-19; 42nd International Conference on Software Engineering : ICSE 2020 ; Conference date: 27-06-2020 Through 19-07-2020",

year = "2020",

doi = "10.1145/3387940.3392265",

language = "English",

pages = "545--548",

booktitle = "Proceedings - 2020 IEEE/ACM 42nd International Conference on Software Engineering Workshops, ICSEW 2020",

publisher = "ACM DL",

}

TY - GEN

T1 - Java Unit Testing Tool Competition - Eighth Round

AU - Devroey, Xavier

AU - Panichella, Sebastiano

AU - Gambi, Alessio

N1 - Virtual/online event due to COVID-19

PY - 2020

Y1 - 2020

N2 - We report on the results of the eighth edition of the Java unit testing tool competition. This year, two tools, EvoSuite and Randoop, were executed on a benchmark with (i) new classes under test, selected from open-source software projects, and (ii) the set of classes from one project considered in the previous edition. We relied on an updated infrastructure for the execution of the different tools and the subsequent coverage and mutation analysis based on Docker containers. We considered two different time budgets for test case generation: one an three minutes. This paper describes our method- ology and statistical analysis of the results, presents the results achieved by the contestant tools and highlights the challenges we faced during the competition.

AB - We report on the results of the eighth edition of the Java unit testing tool competition. This year, two tools, EvoSuite and Randoop, were executed on a benchmark with (i) new classes under test, selected from open-source software projects, and (ii) the set of classes from one project considered in the previous edition. We relied on an updated infrastructure for the execution of the different tools and the subsequent coverage and mutation analysis based on Docker containers. We considered two different time budgets for test case generation: one an three minutes. This paper describes our method- ology and statistical analysis of the results, presents the results achieved by the contestant tools and highlights the challenges we faced during the competition.

KW - tool competition

KW - benchmark

KW - software testing

KW - test case generation

KW - unit testing

KW - Java

KW - JUnit

UR - http://www.scopus.com/inward/record.url?scp=85093076613&partnerID=8YFLogxK

U2 - 10.1145/3387940.3392265

DO - 10.1145/3387940.3392265

M3 - Conference contribution

SP - 545

EP - 548

BT - Proceedings - 2020 IEEE/ACM 42nd International Conference on Software Engineering Workshops, ICSEW 2020

PB - ACM DL

T2 - 42nd International Conference on Software Engineering

Y2 - 27 June 2020 through 19 July 2020

ER -

Java Unit Testing Tool Competition - Eighth Round

Abstract

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this