High Cardinality War Stories

High Cardinality – a term that can derail engineering leaders and software engineers alike. When seeking profound insights from their systems, they often find themselves limited by the cardinality constraints of their observability tools. But why should high cardinality matter, and how has it become an unavoidable challenge when it comes to monitoring software systems on a massive scale? Join me in this captivating talk as we delve deep into the concept of high cardinality, peeling back its layers to reveal its true essence. We'll start by dissecting metrics, understanding their vital role in the observability stack, and using their very structure to lay the groundwork for comprehending high cardinality. Prepare to witness how label sets exponentially multiply before our eyes, driven by our relentless pursuit of answers regarding system behavior and the dynamic nature of metrics. Can we simply turn a blind eye to cardinality and find happiness? I will show you why cardinality has become an unavoidable element in today's software engineering landscape, drawing from my own war stories in the field. From the battles against Noisy Neighbors to the epic Streaming Wars and the pulse of High Cardinality, we'll explore numerous problems that high cardinality can solve. But do our existing observability systems truly support handling high cardinality metrics at the scale we demand without exploding costs? Fear not, as we embark on an exploration of cutting-edge technologies that hold the answers to this burning question. Brace yourself for an enlightening journey that will transform the way you perceive high cardinality and your approach to observing software systems in the modern age.


Speaker

Piyush Verma

Co-Founder and CTO @Last9

Piyush is Co-Founder and CTO at Last9. Previously, Piyush built Datastax-Managed-Cloud (now Constellation), GoLife at Gojek and led Data and Site-Reliability Engineering at startups in India, Bay Area, and Indonesia.

Read more

Session Sponsored By

Last9 provides tools to improve Reliability in large-scale cloud-native environments.

Date

Tuesday Jun 13 / 05:25PM EDT ( 50 minutes )

Location

Williamsburg / Greenpoint

Video

Video is not available

Slides

Slides are not available

Share

From the same track

Session

The Data Behind “Good” Engineering Teams

Tuesday Jun 13 / 01:40PM EDT

Engineering teams have an opportunity to rethink how they measure performance. Productivity and output metrics alone don’t tell the whole story or inspire action. Even DORA metrics — a widely accepted standard for measuring software development performance — leave out some key insights.

Speaker image - Christina  Forney

Christina Forney

VP of Product @Uplevel

Session

Reducing Logging Costs: Making The Impossible Possible

Tuesday Jun 13 / 10:35AM EDT

Engineering leaders are constantly facing the challenge of balancing needing to provide their teams with the best Observability and doing so without wasting significant resources and spend on it.

Speaker image - Josh Hendrick

Josh Hendrick

Senior Solutions Engineer @Rookout

Session

Bring Your Own Cloud – Privacy-Preserving Managed Services

Tuesday Jun 13 / 11:50AM EDT

Companies increasingly rely on cloud-based managed services, which allow them to offload complexity in order to focus on core business activities.

Speaker image - Patrick Angeles

Patrick Angeles

Distributed Data Management @Redpanda

Session

Creating Resilient Distributed Applications that Thrive in the Face of Chaos Monkeys

Tuesday Jun 13 / 02:55PM EDT

 

Speaker image - Viren Baraiya

Viren Baraiya

CTO & Co-Founder @Orkes

Session

A .NET Developer's Journey through OAuth

Tuesday Jun 13 / 04:10PM EDT

Identity is hard. As technologies continue to advance, developers continue to build their own identity solutions, with usernames and passwords. OAuth was created to help address this, and developers are still struggling to navigate through it.

Speaker image - Corey Weathers

Corey Weathers

Developer Relations Lead @Okta/Auth0