Developing an application based on a microservice style architecture bring several benefits. For example, individual microservice applications can be scaled and versioned independently from one another, written in different programming languages and managed by different teams.

While a microservices architecture is commonly applied to create distributed web applications communicating with each other using http, it can be similarly be applied for Stream processing applications that communicate using messaging middleware such as Kafka and RabbitMQ. Spring Cloud Data Flow enables you to orchestrate standalone executable applications to perform stream processing on a variety of runtimes, such as Apache YARN, Kubernetes, Apache Mesos, and PaaS platforms such as CloudFoundry through an extensible abstraction. Data Flow also provides a Unix style pipes and filters DSL that maps each ‘filter’ to executable , so “http | hdfs” represents a simple ingestion stream.

In this session we will create several standalone executable applications, run them via the command line to perform stream processing, and then deploy the distributed application via the DSL to multiple runtimes.