Track: Architectures You've Always Wondered About

Location: Broadway Ballroom North Center, 6th fl.

Day of week: Monday

Ever wonder how they do it? Next-gen architectures from the most admired organizations in software, such as Google, Slack, etc.

Track Host: Randy Shoup

VP Engineering @WeWork

Randy is a 25-year veteran of Silicon Valley, and has worked as a senior technology leader and executive at companies ranging from small startups, to mid-sized places, to eBay and Google. Randy is currently VP Engineering at WeWork in San Francisco. He is particularly passionate about the nexus of culture, technology, and organization.

10:35am - 11:25am

Scaling Infrastructure Engineering at Slack

In August of 2016, I was asked to build Slack’s first Infrastructure engineering organization. The company was a little over 2 years old, and we were approaching the scalability limits of the original infrastructure written by the founders several years prior. Things were starting to break in strange, and unpredictable ways.

Organizations much larger than we had initially envisioned were using Slack. Thousands of developers were building on our external APIs and stressing the system in new and unusual ways. It was taking high double digit seconds to minutes for Slack to load for very large teams, and we wanted to continue growing as fast as we could.

I’ll discuss the architectural and organizational challenges, mistakes and war stories of 2.5 years that followed, including how we:

  • Overcame the initial scalability challenges by building out our caching tier, transitioning many of our internal APIs from broadcast to publish/subscribe and rewrote many parts of our asynchronous job queueing system.
  • Continued to operate our PHP/Hack monolith, but introduced more services, and formalized how we deploy, monitor and build those services.
  • Grew the infrastructure engineering team to a global function with teams around the world.
  • Defined and cultivated an engineering-led culture in a product-led company.
  • Introduced product management, and the evolution of PM in the infrastructure team.
  • Identified key transition points when it was time to hire infrastructure specialists versus generalists.

Julia Grace, Senior Director of Infrastructure Engineering @Slack

11:50am - 12:40pm

Machine-Learned Indexes - Research from Google

Modern data processing systems are designed to be general purpose, in that they can handle a wide variety of different schemas, data types, and data distributions, and aim to provide efficient access and computation over this data. This “one-size-fits-all” nature results in systems that do not take advantage of the unique characteristics of each application, data of the user, or workload.  However, ignored in these old systems’ design: machine learning excels at understanding and adapting to particular datasets.  We present here a vision (with evidence) for the future of data processing systems: through learning models of the application, data, and workload, we can redesign and customize nearly every component of data processing systems.  We will do a deep-dive into understanding how traditional index structures can be reframed as machine learning problems, and that by doing so, and through careful model design and code synthesis, we are able to outperform cache-optimized B-Trees by up to 70% in speed while saving an order-of-magnitude in memory over several real-world data sets.  Building on these same modeling techniques, we find that we can achieve improvements in sorting, multi-dimensional indexing, and query optimization, all areas that have historically been the domain of traditional discrete algorithms and complex systems engineering.

Alex Beutel, Senior Research Scientist @Google
Tim Kraska, Associate Professor @MIT & Visiting Researcher @Google

1:40pm - 2:30pm

Rocket-fueled Engineering & Technology Transformation

WeWork is one of the largest providers of office space in the world, opening an average of 15 new locations around the world every month. For a traditionally physical business, technology has now become an integral part of WeWork’s product offering as well as a key differentiator: software and hardware are now pervasive across the business – from managing real estate planning and construction, to building community applications, to making its buildings smarter. 

In this session, Hugo will go over the platform and architecture behind WeWork’s technology transformation over the past 2.5 years. He will outline some of the unique technology challenges WeWork faces – global systems across China and the rest of the world, hybrid infrastructure between the cloud and on-premise physical buildings, etc. – and describe in detail how WeWork is tackling them.

Hugo Haas, Fellow Engineer / VP, Developer Platform @WeWork

Tracks

Monday, 24 June

Tuesday, 25 June

Wednesday, 26 June

  • Architecting For Failure

    More than just building software, building deployable production ready software in the face of guaranteed failure.

  • 21st Century Languages

    Lessons learned from building languages like Rust, Go-lang, Swift, Kotlin, and more.

  • Building High-Performing Teams

    What “high-performing team” means and how to build one effectively depends on context. This track will share different experiences of building high-performing teams in order to highlight how different contexts lead to different solutions but also what typically stays the same because we’re still dealing with humans trying to work together. How do different forces affect the building of high-performing teams.

  • Software Defined Infrastructure: Kubernetes, Service Meshes, & Beyond

    Deploying, scaling, managing your services is undifferentiated heavy lifting. Hear stories, learn techniques, and dive deep into software infrastructure.

  • High-Performance Computing: Lessons from FinTech & AdTech

    Killing latency and getting the most out of your hardware.