Abstract
Over the past years, parallel dataflow systems have been employed for advanced analytics in the field of data mining where many algorithms are iterative. These systems typically provide fault tolerance by periodically checkpointing the algorithm's state and, in case of failure, restoring a consistent state from a checkpoint. In prior work, we presented an optimistic recovery mechanism that in certain cases eliminates the need to checkpoint the intermediate state of an iterative algorithm. In case of failure, our mechanism uses a compensation function to transit the algorithm to a consistent state, from which the execution can continue and successfully converge. Since this recovery mechanism does not checkpoint any state, it achieves optimal failure-free performance while guaranteeing fault tolerance. In this paper, we demonstrate our recovery mechanism with the Apache Flink data processing engine. During our demonstration, attendees will be able to run graph algorithms and trigger failures to observe the algorithms recovering with compensation functions instead of checkpoints.
Original language | English |
---|---|
Title of host publication | SIGMOD'15 - |
Subtitle of host publication | Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data |
Publisher | Association for Computing Machinery (ACM) |
Pages | 1439-1443 |
Number of pages | 5 |
ISBN (Print) | 978-1-4503-2758-9 |
DOIs | |
Publication status | Published - 27 May 2015 |
Externally published | Yes |
Event | ACM SIGMOD International Conference on Management of Data, SIGMOD 2015 - Melbourne, Australia Duration: 31 May 2015 → 4 Jun 2015 |
Conference
Conference | ACM SIGMOD International Conference on Management of Data, SIGMOD 2015 |
---|---|
Country/Territory | Australia |
City | Melbourne |
Period | 31/05/15 → 4/06/15 |
Keywords
- Fault-tolerance
- Iterative algorithms
- Optimistic recovery