Conference: Jun 26-28, 2017
Workshops: Jun 29-30, 2017
Presentation: Large-Scale Stream Processing with Apache Kafka
Location:
- Salon D
Duration
Day of week:
- Monday
Level:
- Intermediate
Persona:
- Architect
- Data Scientist
Abstract
Most applications continuously transform streams of inputs into streams of outputs. Yet the idea of directly modeling stream processing in applications is just coming into it's own after a few decades on the periphery.
This talk will cover the basic challenges of reliable, distributed, stateful stream processing. It will cover how Apache Kafka was designed to support capturing and processing distributed data streams by building up the basic primitives needed for a stream processing system. It will also introduce Kafka Streams, a lightweight library that applications can embed for expressing stateful stream processing operations.
Finally it will explore how Kafka and Kafka Streams solves practical problems in building scalable and stateful microservices, based on our experience building and scaling Kafka to handle streams that captured hundreds of billions of records per day.
Similar Talks
![](../cdn/farfuture/Buu4Y2SP3gjAv2E8OqaDjLoi9WEewdhWakX_SewOXKw/mtime_1462549232/sites/default/files/styles/spaker_72h/public/mpollack06b2.jpg?itok=B_mRCq07)
![](../cdn/farfuture/KiCEN83ZZZyw9YbU_VGecGnHve57BE2qwwscOClfQiE/mtime_1461245210/sites/default/files/styles/spaker_72h/public/24692920815_2b91b1d56b_o2d8c.jpg?itok=6pW0ypjw)
Tracks
Monday, 13 June
-
Architectures You've Always Wondered About
Case studies from: Google, Linkedin, Alibaba, Twitter, and more...
-
Stream Processing @ Scale
Technologies and techniques to handle ever increasing data streams
-
Culture As Differentiator
Stories of companies and team for whom engineering culture is a differentiator - in delivering faster, in attracting better talent, and in making their businesses more successful.
-
Practical DevOps for Cloud Architectures
Real-world lessons and practices that enable the devops nirvana of operating what you build
-
Incredible Power of an Open-Sourced .NET
.NET is more than you may think. From Rx to C# 7 designed in the open, learn more about the power of open source .NET
-
Sponsored Solutions Track 1
Tuesday, 14 June
-
Better than Resilient: Antifragile
Failure is a constant in production systems, learn how to wield it to your advantage to build more robust systems.
-
Innovations in Java and the Java Ecosystem
Cutting Edge Java Innovations for the Real World
-
Modern CS in the Real World
Real-world Industry adoption of modern CS ideas
-
Containers: From Dev to Prod
Beyond the buzz and into the how and why of running containers in production
-
Security War Stories
Expert-level security track led by well known and respected leaders in the field
-
Sponsored Solutions Track 2
Wednesday, 15 June
-
Microservices and Monoliths
Practical lessons on services. Asks the question when and when to NOT go with Microservices?
-
Modern API Architecture - Tools, Methods, Tactics
API-based application development, and the tooling and techniques to support effectively working with APIs in the small or at scale. Using internal and external APIs
-
Commoditized Machine Learning
Barriers to entry for applied ML are lower than ever before, jumpstart your journey
-
Full Stack JavaScript
Browser, server, devices - JavaScript is everywhere
-
Optimizing Yourself
Keeping life in balance is always a challenge. Learning lifehacks
-
Sponsored Solutions Track 3