Track: Stream Processing at Large
Day of week:
The software industry has learned that the world’s data can be represented as unbounded queues of changes. It can be sliced into sliding windows. It can be aggregated, rolled up, and analyzed. We can choose a number of ways to do this work such as using Kafka Streams or Spark Streaming. We can opt for Apache Beam, Storm, Samza, Flume, or Flink. We have a large pool of options on which we can build powerful systems, but there is accidental complexity lurking in any of the choices:
- What if I need to rebuild all the data?
- How do I know when my system is not healthy?
- How do I reason about time in this system?
- What if things arrive out of order?
- How do I know things have arrived?
This track walks through uses of streaming technologies at large, the problems encountered, and how teams are coping with the state of this new world. As we approach maturity in streaming systems the companies using these systems are growing ecosystems and best practices around building and operating them. They are discovering new ways to reason about monitoring, testing, performance, and failure. This track is an opportunity to learn from their experiences.
by Shriya Arora
Senior Data Engineer @Netflix
Streaming applications have historically been complex to design and implement because of the significant infrastructure investment. However, recent active developments in various streaming platforms provide an easy transition to stream processing, and enable analytics applications/experiments to consume near real-time data without massive development cycles.
This talk will cover the experiences Netflix’s Personalization Data team had in stream processing unbounded datasets. The...
by Sean Cribbs
Software Engineer @Comcast
In the midst of building a multi-datacenter, multi-tenant instrumentation and visibility system, we arrived at stream processing as an alternative to storing, forwarding, and post-processing metrics as traditional systems do. However, the streaming paradigm is alien to many engineers and sysadmins who are used to working with "wall-of-graphs" dashboards, predefined aggregates, and point-and-click alert configuration.
Taking inspiration from REPLs, literate programming, and DevOps...
Monday, 26 June
High Velocity Dev Teams
Working Smarter as a team. Improving value delivery of engineers. Lean and Agile principles.
Innovations in Fintech
Technology, tools and techniques supporting modern financial services.
Java - Propelling the Ecosystem Forward
Lessons from Java 8, prepping for Java 9, and looking ahead at Java 10. Innovators in Java.
Microservices: Patterns & Practices
Practical experiences and lessons with Microservices.
Modern Clientside Apps
Reactive, cross platform, progressive - webapp tech today.
Tuesday, 27 June
Architectures You've Always Wondered about
Case studies from the most relevant names in software.
Building Security Infrastructure
How our industry is being attacked and what you can do about it.
Chaos & Resilience
Failures, edge cases and how we're embracing them.
Developer Experience: Level up your engineering effectiveness
Trends, tools and projects that we're using to maximally empower your developers.
Stream Processing at Large
Rapidly moving data at scale.
Wednesday, 28 June
Immutable Infrastructures: Orchestration, Serverless, and More
What's next in infrastructure. How cloud function like lambda are making their way into production.
Machine Learning 2.0
Machine Learning 2.0, Deep Learning & Deep Learning Datasets.
Modern CS in the Real World
Applied, practical, & real-world dive into industry adoption of modern CS.
Next Gen APIs: Designs, Protocols, and Evolution
Practical deep-dives into public and internal API design, tooling and techniques for evolving them, and binary and graph-based protocols.
Maximizing your impact as an engineer, as a leader, and as a person.