Cloud computing is a new paradigm for using ICT services—only when needed and for as long as needed, and paying only for service actually consumed. Benchmarking the increasingly many cloud services is crucial for market growth and perceived fairness, and for service design and tuning. In this work, we propose a generic architecture for benchmarking cloud services. Motivated by recent demand for data-intensive ICT services, and in particular by processing of large graphs, we adapt the generic architecture to Graphalytics, a benchmark for distributed and GPU-based graph analytics platforms. Graphalytics focuses on the
dependence of performance on the input dataset, on the analytics algorithm,
and on the provisioned infrastructure. The benchmark provides components for platform configuration, deployment, and monitoring, and has been tested for a variety of platforms. We also propose a new challenge for the process of benchmarking data-intensive services, namely the inclusion of the data-processing algorithm in the system under test; this increases significantly the relevance of benchmarking results, albeit, at the cost of increased benchmarking duration.
Original languageEnglish
Title of host publicationBig Data Benchmarlking
Subtitle of host publication5th International Workshop, WBDB 2014, Revised Selected Papers
EditorsTilmann Rabl, Kai Sachs, Meikel Poess, Chaitanya Baru, Hans-Arno Jacobson
Place of PublicationCham
Number of pages23
ISBN (Electronic)978-3-319-20233-4
ISBN (Print)978-331920232-7
Publication statusPublished - 2014
EventBig Data Benchmarking: 5th International Workshop, WBDB 2014 - Cham, Potsdam, Germany
Duration: 5 Aug 20146 Aug 2014
Conference number: 5

Publication series

NameLecture Notes in Computer Science
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


WorkshopBig Data Benchmarking
Abbreviated titleWBDB

ID: 1878014