Presentation: Datadog: A Real Time Metrics Database for One Quadrillion Points/Day

Track: Data Engineering for the Bold

Location: Majestic Complex, 6th fl.

Duration: 4:10pm - 5:00pm

Day of week: Tuesday

Share this on:

Abstract

In the course of its 8 years of existence, Datadog has grown its real time metrics systems that collect, process, and visualize data to the point they now handle trillions of points per day. This has been based on an architecture combining open source technologies such as Apache Cassandra, Kafka and PostgreSQL with a lot of in-house software, particularly for in-memory data storing and querying; especially for efficiently computing distribution metrics. In this talk Datadog's VP Metrics and Alerts Ian Nowland and Director of Distribution Metrics Joel Barciauskas, will speak to the challenges we face, how the architecture has evolved to cope with them, and what we are looking to in the future as we architect for a quadrillion points per day.

Speaker: Ian Nowland

VP Engineering Metrics and Alerting @datadoghq

Ian Nowland is the VP Engineering Metrics and Alerting at Datadog. Before that he was SVP Engineering Manager of the Compute Platform at Two Sigma, and he spent 8 years in AWS where his major achievement was building the team that shipped the first three generations of the EC2 Nitro platform.

Find Ian Nowland at

Speaker: Joel Barciauskas

Director of Engineering, Distribution Metrics

Joel is an experienced lead engineer and technical manager with an extensive engineering and technical consulting background. He currently leads Datadog's distribution metrics team, providing accurate, low latency percentile measures for customers across their infrastructure.

Find Joel Barciauskas at

Tracks

Monday, 24 June

Tuesday, 25 June

Wednesday, 26 June