Speaker
Abstract
High Cardinality – a term that can derail engineering leaders and software engineers alike. When seeking profound insights from their systems, they often find themselves limited by the cardinality constraints of their observability tools. But why should high cardinality matter, and how has it become an unavoidable challenge when it comes to monitoring software systems on a massive scale? Join me in this captivating talk as we delve deep into the concept of high cardinality, peeling back its layers to reveal its true essence. We'll start by dissecting metrics, understanding their vital role in the observability stack, and using their very structure to lay the groundwork for comprehending high cardinality. Prepare to witness how label sets exponentially multiply before our eyes, driven by our relentless pursuit of answers regarding system behavior and the dynamic nature of metrics. Can we simply turn a blind eye to cardinality and find happiness? I will show you why cardinality has become an unavoidable element in today's software engineering landscape, drawing from my own war stories in the field. From the battles against Noisy Neighbors to the epic Streaming Wars and the pulse of High Cardinality, we'll explore numerous problems that high cardinality can solve. But do our existing observability systems truly support handling high cardinality metrics at the scale we demand without exploding costs? Fear not, as we embark on an exploration of cutting-edge technologies that hold the answers to this burning question. Brace yourself for an enlightening journey that will transform the way you perceive high cardinality and your approach to observing software systems in the modern age.
Sponsored session
QCon New York 2023 is a three day conference for senior software engineers, architects and team leads. An international program committee of working engineers selects every session. Patterns and practices, not products and pitches.
Part of the track
Sponsored Solution Track IFrom the same track
Tuesday 13 June
10:35 Williamsburg / Greenpoint Sponsored Reducing Logging Costs: Making The Impossible Possible Josh Hendrick Senior Solutions Engineer @Rookout Engineering leaders are constantly facing the challenge of balancing needing to provide their teams with the best Observability and doing so without wasting significant resources and spend on it. 11:50 Williamsburg / Greenpoint Sponsored Bring Your Own Cloud – Privacy-Preserving Managed Services Patrick Angeles Distributed Data Management @Redpanda Companies increasingly rely on cloud-based managed services, which allow them to offload complexity in order to focus on core business activities. 13:40 Williamsburg / Greenpoint Sponsored The Data Behind “Good” Engineering Teams Christina Forney VP of Product @Uplevel Engineering teams have an opportunity to rethink how they measure performance. Productivity and output metrics alone don’t tell the whole story or inspire action. Even DORA metrics — a widely accepted standard for measuring software development performance — leave out some key insights. 14:55 Williamsburg / Greenpoint Sponsored Creating Resilient Distributed Applications that Thrive in the Face of Chaos Monkeys Viren Baraiya CTO & Co-Founder @Orkes 16:10 Williamsburg / Greenpoint Sponsored A .NET Developer's Journey through OAuth Corey Weathers Developer Relations Lead @Okta/Auth0 Identity is hard. As technologies continue to advance, developers continue to build their own identity solutions, with usernames and passwords. OAuth was created to help address this, and developers are still struggling to navigate through it. 17:25 Williamsburg / Greenpoint Sponsored High Cardinality War Stories Piyush Verma Co-Founder and CTO @Last9 High Cardinality – a term that can derail engineering leaders and software engineers alike. When seeking profound insights from their systems, they often find themselves limited by the cardinality constraints of their observability tools.