Presentation: A Microservice Based Architecture for Stream Processing using Spring Cloud Data Flow



1:40pm - 2:30pm

Day of week:



Developing an application based on a microservice style architecture bring several benefits. For example, individual microservice applications can be scaled and versioned independently from one another, written in different programming languages and managed by different teams.

While a microservices architecture is commonly applied to create distributed web applications communicating with each other using http, it can be similarly be applied for Stream processing applications that communicate using messaging middleware such as Kafka and RabbitMQ. Spring Cloud Data Flow enables you to orchestrate standalone executable applications to perform stream processing on a variety of runtimes, such as Apache YARN, Kubernetes, Apache Mesos, and PaaS platforms such as CloudFoundry through an extensible abstraction. Data Flow also provides a Unix style pipes and filters DSL that maps each ‘filter’ to executable , so “http | hdfs” represents a simple ingestion stream.

In this session we will create several standalone executable applications, run them via the command line to perform stream processing, and then deploy the distributed application via the DSL to multiple runtimes.

Speaker: Mark Pollack

Senior Staff Engineer @Pivotal

I am a contributor to many Spring projects, starting back in 2003 with the Spring Framework. Most notably, I founded the Spring.NET and Spring Data projects and now co-lead the Spring Data Flow project.

Find Mark Pollack at


Monday, 13 June

Tuesday, 14 June

Wednesday, 15 June