High Cardinality War Stories

QCon New York 2023

Sponsored session

High Cardinality War Stories

Tuesday Jun 13 / 05:25PM EDT, Williamsburg / Greenpoint

Abstract

High Cardinality – a term that can derail engineering leaders and software engineers alike. When seeking profound insights from their systems, they often find themselves limited by the cardinality constraints of their observability tools. But why should high cardinality matter, and how has it become an unavoidable challenge when it comes to monitoring software systems on a massive scale? Join me in this captivating talk as we delve deep into the concept of high cardinality, peeling back its layers to reveal its true essence. We'll start by dissecting metrics, understanding their vital role in the observability stack, and using their very structure to lay the groundwork for comprehending high cardinality. Prepare to witness how label sets exponentially multiply before our eyes, driven by our relentless pursuit of answers regarding system behavior and the dynamic nature of metrics. Can we simply turn a blind eye to cardinality and find happiness? I will show you why cardinality has become an unavoidable element in today's software engineering landscape, drawing from my own war stories in the field. From the battles against Noisy Neighbors to the epic Streaming Wars and the pulse of High Cardinality, we'll explore numerous problems that high cardinality can solve. But do our existing observability systems truly support handling high cardinality metrics at the scale we demand without exploding costs? Fear not, as we embark on an exploration of cutting-edge technologies that hold the answers to this burning question. Brace yourself for an enlightening journey that will transform the way you perceive high cardinality and your approach to observing software systems in the modern age.

Sponsored session

Session sponsored by

Last9 provides tools to improve Reliability in large-scale cloud-native environments.

76% senior dev or higher
1:11 speaker ratio
60+ practitioners

QCon New York 2023 is a three day conference for senior software engineers, architects and team leads. An international program committee of working engineers selects every session. Patterns and practices, not products and pitches.

Share

From the same track

Tuesday 13 June

10:35 Williamsburg / Greenpoint Sponsored Reducing Logging Costs: Making The Impossible Possible Josh Hendrick Senior Solutions Engineer @Rookout 11:50 Williamsburg / Greenpoint Sponsored Bring Your Own Cloud – Privacy-Preserving Managed Services Patrick Angeles Distributed Data Management @Redpanda 13:40 Williamsburg / Greenpoint Sponsored The Data Behind “Good” Engineering Teams Christina Forney VP of Product @Uplevel 14:55 Williamsburg / Greenpoint Sponsored Creating Resilient Distributed Applications that Thrive in the Face of Chaos Monkeys Viren Baraiya CTO & Co-Founder @Orkes 16:10 Williamsburg / Greenpoint Sponsored A .NET Developer's Journey through OAuth Corey Weathers Developer Relations Lead @Okta/Auth0 17:25 Williamsburg / Greenpoint Sponsored High Cardinality War Stories Piyush Verma Co-Founder and CTO @Last9