Presentation: Scaling Infrastructure Engineering at Slack

Track: Architectures You've Always Wondered About

Location: Broadway Ballroom North, 6th fl.

Duration: 10:35am - 11:25am

Day of week: Monday

Level: Advanced

Persona: Architect

Share this on:

Abstract

In August of 2016, I was asked to build Slack’s first Infrastructure engineering organization. The company was a little over 2 years old, and we were approaching the scalability limits of the original infrastructure written by the founders several years prior. Things were starting to break in strange, and unpredictable ways.

Organizations much larger than we had initially envisioned were using Slack. Thousands of developers were building on our external APIs and stressing the system in new and unusual ways. It was taking high double digit seconds to minutes for Slack to load for very large teams, and we wanted to continue growing as fast as we could.

I’ll discuss the architectural and organizational challenges, mistakes and war stories of 2.5 years that followed, including how we:

  • Overcame the initial scalability challenges by building out our caching tier, transitioning many of our internal APIs from broadcast to publish/subscribe and rewrote many parts of our asynchronous job queueing system.
  • Continued to operate our PHP/Hack monolith, but introduced more services, and formalized how we deploy, monitor and build those services.
  • Grew the infrastructure engineering team to a global function with teams around the world.
  • Defined and cultivated an engineering-led culture in a product-led company.
  • Introduced product management, and the evolution of PM in the infrastructure team.
  • Identified key transition points when it was time to hire infrastructure specialists versus generalists.

Speaker: Julia Grace

Senior Director of Infrastructure Engineering @Slack

Julia loves solving challenging engineering problems at scale, growing businesses, and leading teams. She’s currently a Senior Director of Product Engineering at Slack focused on building network effects into Slack through shared channels. Prior to joining product engineering she built the infrastructure team at Slack, growing it from from 10 to 100 engineers in 3 offices in 2 years. She excels in high velocity environments especially during hyper growth: Slack engineering headcount grew from 100 to over 600 during her 3.5 years there.
 
She advises early and mid-stage startups, having extensive experience not only raising venture capital funding (including from top tier investors such as Andreessen Horowitz), and sold her previous company, Tindie, where she was the co-founder and CTO. She has also founded and sat on several advisory boards for startups and as well as large, multi-billion dollar non-profits. She holds a BS with Honors and MS in Computer Science from the University of North Carolina where her research focused on operating systems, including building a distributed system that allowed peer-to-peer data sharing from internet browser caches in low connectivity. She is an avid athlete and former collegiate rower, always trying to squeeze a run in (including chasing her young daughter)!

Find Julia Grace at

Similar Talks

Psychologically Safe Process Evolution in a Flat Structure

Director of Software Development @Hunter_Ind

Chris Lucian

Let's talk locks!

Software Engineer @Samsara

Kavya Joshi

Graceful Degradation as a Feature

Director of Product @GremlinInc

Lorne Kligerman

What Breaks Our Systems: A Taxonomy of Black Swans

Site Reliability Engineer @Slack, Contributor to Seeking SRE, & SRECon Steering Committee

Laura Nolan

Liberating Structures at Capital One

Agile Coach, Engineering @CapitalOne

Greg Myers

Making 'npm install' Safe

Software Engineer @agoric

Kate Sills

Tracks

Monday, 24 June

Tuesday, 25 June

Wednesday, 26 June