Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Scientific › peer-review
An Elasticity Study of Distributed Graph Processing. / Au, Sietse; Uta, Alexandru; Ilyushkin, Alexey; Iosup, Alexandru.
2018 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID). ed. / L. O'Conner. Piscataway, NJ : IEEE, 2018. p. 382-383.Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Scientific › peer-review
}
TY - GEN
T1 - An Elasticity Study of Distributed Graph Processing
AU - Au, Sietse
AU - Uta, Alexandru
AU - Ilyushkin, Alexey
AU - Iosup, Alexandru
N1 - Accepted author manuscript
PY - 2018
Y1 - 2018
N2 - Graphs are a natural fit for modeling concepts used in solving diverse problems in science, commerce, engineering, and governance. Responding to the variety of graph data and algorithms, many parallel and distributed graph processing systems exist. However, until now these platforms use a static model of deployment: they only run on a pre-defined set of machines. This raises many conceptual and pragmatic issues, including misfit with the highly dynamic nature of graph processing, and could lead to resource waste and high operational costs. In contrast, in this work we explore a dynamic model of deployment. We first characterize workload dynamicity, beyond mere active-vertex variability. Then, to conduct an in-depth elasticity study of distributed graph processing, we build a prototype, JoyGraph, which is the first such system that implements complex, policy-based, and fine-grained elasticity. Using the state-of-the-art LDBC Graphalytics benchmark and the SPEC Cloud Group's elasticity metrics, we show the benefits of elasticity in graph processing: (i) improved resource utilization, (ii) reduced operational costs, and (iii) aligned operation-workload dynamicity. Furthermore, we explore the cost of elasticity in graph processing. We identify a key drawback: although elasticity does not degrade application throughput, graph-processing workloads are sensitive to data movement while leasing or releasing resources.
AB - Graphs are a natural fit for modeling concepts used in solving diverse problems in science, commerce, engineering, and governance. Responding to the variety of graph data and algorithms, many parallel and distributed graph processing systems exist. However, until now these platforms use a static model of deployment: they only run on a pre-defined set of machines. This raises many conceptual and pragmatic issues, including misfit with the highly dynamic nature of graph processing, and could lead to resource waste and high operational costs. In contrast, in this work we explore a dynamic model of deployment. We first characterize workload dynamicity, beyond mere active-vertex variability. Then, to conduct an in-depth elasticity study of distributed graph processing, we build a prototype, JoyGraph, which is the first such system that implements complex, policy-based, and fine-grained elasticity. Using the state-of-the-art LDBC Graphalytics benchmark and the SPEC Cloud Group's elasticity metrics, we show the benefits of elasticity in graph processing: (i) improved resource utilization, (ii) reduced operational costs, and (iii) aligned operation-workload dynamicity. Furthermore, we explore the cost of elasticity in graph processing. We identify a key drawback: although elasticity does not degrade application throughput, graph-processing workloads are sensitive to data movement while leasing or releasing resources.
U2 - 10.1109/CCGRID.2018.00062
DO - 10.1109/CCGRID.2018.00062
M3 - Conference contribution
SN - 978-1-5386-5816-1
SP - 382
EP - 383
BT - 2018 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID)
A2 - O'Conner, L.
PB - IEEE
CY - Piscataway, NJ
ER -
ID: 46652034