Operational stream processing: Towards scalable and consistent event-driven applications

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

11 Citations (Scopus)
872 Downloads (Pure)

Abstract

In the last decade we are witnessing a widespread adoption of architectural styles such as microservices, for building event-driven software applications and deploying them in cloud infrastructures. Such services favor the separation of a database into independent silos of data, each of which is owned entirely by a single service. As a result, traditional oltp systems no longer fit the architectural picture and developers often turn to ad-hoc solutions that rarely support acid transaction consistency. At the same time, we are witnessing the gradual maturation of distributed streaming dataflow systems. These systems nowadays have departed from the mere analysis of streaming windows and complex-event processing, employing sophisticated methods for managing state, keeping it consistent, and ensuring exactly-once processing guarantees in the presence of failures. The goal of this paper is threefold. First, we illustrate the requirements of stateful software services in terms of consistency and scalability. Second, we present how well existing solutions meet those requirements. Finally, we outline a set of challenging problems and propose research directions for enabling event-driven applications to be developed on top of streaming dataflow systems. We strongly believe that streaming dataflows can have a central place in service-oriented architectures, taking over the execution of acid transactions, ensuring message delivery and processing, in order to perform scalable execution of services.

Original languageEnglish
Title of host publicationAdvances in Database Technology - EDBT 2019
Subtitle of host publication22nd International Conference on Extending Database Technology, Proceedings
EditorsBerthold Reinwald, Helena Galhardas, Irini Fundulaki, Melanie Herschel, Zoi Kaoudi, Carsten Binnig
PublisherOpenProceedings.org
Pages682-685
Number of pages4
Volume2019-March
ISBN (Electronic)978-3-89318-081-3
DOIs
Publication statusPublished - 2019
Event22nd International Conference on Extending Database Technology, EDBT 2019 - Lisbon, Portugal
Duration: 26 Mar 201929 Mar 2019

Conference

Conference22nd International Conference on Extending Database Technology, EDBT 2019
Country/TerritoryPortugal
CityLisbon
Period26/03/1929/03/19

Fingerprint

Dive into the research topics of 'Operational stream processing: Towards scalable and consistent event-driven applications'. Together they form a unique fingerprint.

Cite this