Standard

Quality Control in Crowdsourcing : A Survey of Quality Attributes, Assessment Techniques, and Assurance Actions. / Daniel, Florian; Kucherbaev, Pavel; Cappiello, Cinzia; Benatallah, Boualem; Allahbakhsh, Mohammad.

In: ACM Computing Surveys: the survey and tutorial journal of the ACM, Vol. 51, No. 1, 01.01.2018, p. 7:1-7:40.

Research output: Scientific - peer-reviewArticle

Harvard

Daniel, F, Kucherbaev, P, Cappiello, C, Benatallah, B & Allahbakhsh, M 2018, 'Quality Control in Crowdsourcing: A Survey of Quality Attributes, Assessment Techniques, and Assurance Actions' ACM Computing Surveys: the survey and tutorial journal of the ACM, vol 51, no. 1, pp. 7:1-7:40. DOI: 10.1145/3148148

APA

Daniel, F., Kucherbaev, P., Cappiello, C., Benatallah, B., & Allahbakhsh, M. (2018). Quality Control in Crowdsourcing: A Survey of Quality Attributes, Assessment Techniques, and Assurance Actions. ACM Computing Surveys: the survey and tutorial journal of the ACM, 51(1), 7:1-7:40. DOI: 10.1145/3148148

Vancouver

Daniel F, Kucherbaev P, Cappiello C, Benatallah B, Allahbakhsh M. Quality Control in Crowdsourcing: A Survey of Quality Attributes, Assessment Techniques, and Assurance Actions. ACM Computing Surveys: the survey and tutorial journal of the ACM. 2018 Jan 1;51(1):7:1-7:40. Available from, DOI: 10.1145/3148148

Author

Daniel, Florian ; Kucherbaev, Pavel ; Cappiello, Cinzia ; Benatallah, Boualem ; Allahbakhsh, Mohammad. / Quality Control in Crowdsourcing : A Survey of Quality Attributes, Assessment Techniques, and Assurance Actions. In: ACM Computing Surveys: the survey and tutorial journal of the ACM. 2018 ; Vol. 51, No. 1. pp. 7:1-7:40

BibTeX

@article{035a816889d74118bf88dd50a786bdfc,
title = "Quality Control in Crowdsourcing: A Survey of Quality Attributes, Assessment Techniques, and Assurance Actions",
abstract = "Crowdsourcing enables one to leverage on the intelligence and wisdom of potentially large groups of individuals toward solving problems. Common problems approached with crowdsourcing are labeling images, translating or transcribing text, providing opinions or ideas, and similar—all tasks that computers are not good at or where they may even fail altogether. The introduction of humans into computations and/or everyday work, however, also poses critical, novel challenges in terms of quality control, as the crowd is typically composed of people with unknown and very diverse abilities, skills, interests, personal objectives, and technological resources. This survey studies quality in the context of crowdsourcing along several dimensions, so as to define and characterize it and to understand the current state of the art. Specifically, this survey derives a quality model for crowdsourcing tasks, identifies the methods and techniques that can be used to assess the attributes of the model, and the actions and strategies that help prevent and mitigate quality problems. An analysis of how these features are supported by the state of the art further identifies open issues and informs an outlook on hot future research directions.",
keywords = "Crowdsourcing, quality model, attributes, assessment, assurance",
author = "Florian Daniel and Pavel Kucherbaev and Cinzia Cappiello and Boualem Benatallah and Mohammad Allahbakhsh",
year = "2018",
month = "1",
doi = "10.1145/3148148",
volume = "51",
pages = "7:1--7:40",
journal = "ACM Computing Surveys: the survey and tutorial journal of the ACM",
issn = "0360-0300",
publisher = "Association for Computing Machinery (ACM)",
number = "1",

}

RIS

TY - JOUR

T1 - Quality Control in Crowdsourcing

T2 - ACM Computing Surveys: the survey and tutorial journal of the ACM

AU - Daniel,Florian

AU - Kucherbaev,Pavel

AU - Cappiello,Cinzia

AU - Benatallah,Boualem

AU - Allahbakhsh,Mohammad

PY - 2018/1/1

Y1 - 2018/1/1

N2 - Crowdsourcing enables one to leverage on the intelligence and wisdom of potentially large groups of individuals toward solving problems. Common problems approached with crowdsourcing are labeling images, translating or transcribing text, providing opinions or ideas, and similar—all tasks that computers are not good at or where they may even fail altogether. The introduction of humans into computations and/or everyday work, however, also poses critical, novel challenges in terms of quality control, as the crowd is typically composed of people with unknown and very diverse abilities, skills, interests, personal objectives, and technological resources. This survey studies quality in the context of crowdsourcing along several dimensions, so as to define and characterize it and to understand the current state of the art. Specifically, this survey derives a quality model for crowdsourcing tasks, identifies the methods and techniques that can be used to assess the attributes of the model, and the actions and strategies that help prevent and mitigate quality problems. An analysis of how these features are supported by the state of the art further identifies open issues and informs an outlook on hot future research directions.

AB - Crowdsourcing enables one to leverage on the intelligence and wisdom of potentially large groups of individuals toward solving problems. Common problems approached with crowdsourcing are labeling images, translating or transcribing text, providing opinions or ideas, and similar—all tasks that computers are not good at or where they may even fail altogether. The introduction of humans into computations and/or everyday work, however, also poses critical, novel challenges in terms of quality control, as the crowd is typically composed of people with unknown and very diverse abilities, skills, interests, personal objectives, and technological resources. This survey studies quality in the context of crowdsourcing along several dimensions, so as to define and characterize it and to understand the current state of the art. Specifically, this survey derives a quality model for crowdsourcing tasks, identifies the methods and techniques that can be used to assess the attributes of the model, and the actions and strategies that help prevent and mitigate quality problems. An analysis of how these features are supported by the state of the art further identifies open issues and informs an outlook on hot future research directions.

KW - Crowdsourcing

KW - quality model

KW - attributes

KW - assessment

KW - assurance

U2 - 10.1145/3148148

DO - 10.1145/3148148

M3 - Article

VL - 51

SP - 7:1-7:40

JO - ACM Computing Surveys: the survey and tutorial journal of the ACM

JF - ACM Computing Surveys: the survey and tutorial journal of the ACM

SN - 0360-0300

IS - 1

ER -

ID: 36754687