warning icon QCon New York 2020 has been canceled. See our current virtual and in-person events.

Track: Mission Critical Data Engineering

Day of week: Tuesday

In the old days, data engineering was all about getting data from a transactional data store to a data warehouse, and enabling business analysts to create their reports.

But we’ve come a long way since - Data is now used to power mission critical production applications, create powerful network effects, predict user behavior and drive cutting-edge AI. The tools and techniques used by data engineering professionals changed as well - we are seeing more microservices, more kubernetes, more meshes, more real-time, more variety of data and more interaction with users. 

In this track we’ll explore the ways in which data engineering has changed in the last few years and became key to the success of products and companies - new requirements, new techniques and new solutions. 

Track Host: Gwen Shapira

Engineering manager @Confluent, Apache Kafka PMC, author of Kafka the Definitive Guide

Gwen is a principal data architect at Confluent helping customers to achieve success with their Apache Kafka implementation. She has 15 years of experience working with code and customers to build scalable data architectures, integrating microservices, relational and big data technologies. She currently specializes in building real-time reliable data processing pipelines using Apache Kafka. Gwen is an author of “Kafka - the Definitive Guide”, "Hadoop Application Architectures", and a frequent presenter at industry conferences. Gwen is also a committer on the Apache Kafka and Apache Sqoop projects. When Gwen isn't coding or building data pipelines, you can find her pedaling on her bike exploring the roads and trails of California, and beyond.