You are viewing content from a past/completed QCon

Presentation: Canopy: Scalable Distributed Tracing & Analysis @Facebook

Track: Architectures You've Always Wondered About

Location: Broadway Ballroom North, 6th fl.

Duration: 11:50am - 12:40pm

Day of week: Wednesday

Level: Intermediate

Persona: Architect, Developer

Share this on:

Abstract

How do you understand the performance of a request that is executed in a large-scale system, potentially fanning out across thousands of machines and services? To answer this question at Facebook, we built a distributed tracing framework, Canopy, which has provided visibility into an otherwise intractable problem. 

In this talk we present Canopy, Facebook’s performance and efficiency tracing infrastructure. Canopy recoards causally related events across the end-to-end execution path of requests, including from browsers, mobile applications, and backend services. Canopy processes traces in near real-time, derives user-specified features, and outputs to datasets that aggregate across billions of requests. At Facebook, Canopy is used to query and analyze performance and efficiency data in real-time.   

Canopy addresses three challenges we have encountered: (1) supporting the range of execution and performance models used by different components of the Facebook stack; (2) supporting interactive ad-hoc and real-time analysis of trace data; and (3) operating at massive scale - Canopy currently records and processes over 1 billion traces per day.   

We conclude by discussing lessons learned applying Canopy to a wide range of use cases at Facebook and present case studies of its use in solving various performance and efficiency challenges

Speaker: Haozhe Gao

Software Engineer @Facebook

Edison is a Software Engineer at Facebook on the Canopy team, Facebook's distributed tracing framework. Prior to joining Facebook, he interned at Palantir. Edison graduated from Carnegie Mellon University with a BS in Mathematics and a BS in Computer Science.

Find Haozhe Gao at

Speaker: Joe O’Neill

Software Engineer @Facebook

Joe is a Software Engineer at Facebook on the Canopy team, Facebook's distributed tracing framework. Prior to joining Facebook, he interned at Microsoft. Joe graduated from Northeastern University with a BS in Computer Science and Business Administration.

Find Joe O’Neill at

Tracks