Presentation: Large-Scale Stream Processing with Apache Kafka



4:10pm - 5:00pm

Day of week:




Most applications continuously transform streams of inputs into streams of outputs. Yet the idea of directly modeling stream processing in applications is just coming into it's own after a few decades on the periphery.

This talk will cover the basic challenges of reliable, distributed, stateful stream processing. It will cover how Apache Kafka was designed to support capturing and processing distributed data streams by building up the basic primitives needed for a stream processing system. It will also introduce Kafka Streams, a lightweight library that applications can embed for expressing stateful stream processing operations.

Finally it will explore how Kafka and Kafka Streams solves practical problems in building scalable and stateful microservices, based on our experience building and scaling Kafka to handle streams that captured hundreds of billions of records per day.

Speaker: Neha Narkhede

Co-Creator Apache Kafka/Co-Founder & Head of Engineering @Confluent

Neha Narkhede is co-founder and CTO at Confluent. She is also the co-creator of Apache Kafka and serves as a PMC member and committer for the project. Previously, she was responsible for LinkedIn’s petabyte scale streaming infrastructure supporting hundreds of billions of events per day. Prior to that, she has worked on search within the database at Oracle and holds a Masters in Computer Science from Georgia Tech.

Find Neha Narkhede at


Monday, 13 June

Tuesday, 14 June

Wednesday, 15 June