Presentation: Lessons Learned on Uber's Journey into Microservices

Location:

Duration

Duration: 
2:55pm - 3:45pm

Day of week:

Level:

Persona:

Key Takeaways

  • Hear stories (good and bad) as a rapid growth company broke up a monolith into a series of Microservices.
  • Learn practices and lessons that will save you time and money if you get right before breaking up your monolith.
  • Understand the story around scalability and moving your architecture to Microservices.

Abstract

Uber had a monolith endearingly named "API" - which at the time comprised of the bulk of Uber's backend processing code. As this service was getting unwieldy to maintain - for scalability and a variety of reason - we launched on a journey to break this up into Microservices. Thus began "Project Darwin". Little did we realize what we were in for. Turns out that migrations are not easy, and take a long, long time. At Uber, the physical world depends on our service, and as such, we hold ourselves to a high standard for availability. That means when we build new systems, there are many phases of validating the new system before production traffic can be rerouted to it. With this talk, we hope to share our learning with you to help you when you undergo a large migration of your own!

Interview

Question: 
What is your role today?
Answer: 
I am a developer on Uber’s money team, which consists of collecting money from Uber riders after taking trips and then paying out the driver partners. I have been involved in both sides since joining Uber in January last year.
Question: 
Tell me a bit about the architecture your team uses.
Answer: 
Most of our code has been in a monolith up until this year, and we’ve broken it apart and spun up 15 to 20 services within the money team organization. Historically all of our code has been Python but the new microservices are starting to emerge in Go.
Question: 
What is the main topic of your talk?
Answer: 
This migration has been a huge undertaking and we’ve learned several things along the way. I am going to tell the war story of how this migration has gone, what we have learned, and how it could have gone more smoothly.
For example, the monolith held most of Uber’s backend code, but we also have many consumers that aren’t part of this monolith. Our migration has significantly impacted them as well since they have to migrate from calling what was once the monolith to a several new microservices. One thing that we have learned is to align early with your consumers. Let them know what is coming and how you are going to help them through the migration.
It took a lot of effort to convince our consumers later to undergo this migration, because for them, there is not much benefit. Many product teams are focused on improving their product -- they don't directly see much value added from changing one single API call into calls to 4, 5 or 10 microservices. Getting buy in from them early on will speed up this process a lot.
Also, a lot of difficulty came with not knowing who our consumers were. Building tooling up front would have helped consumer migration carry out more smoothly. We have built tooling once we realized it was necessary, but it would have helped if we had built these tools before we started the migration.
For example, a consumer may have previously made one API call that would extract data from many data models. Now, one of those data models has moved out into a microservice, but the consumer still needs all that data conglomerated into one response. Thus, we have now built a tool that splits the endpoint that used to touch all the data models into separate endpoints, with one of them talking to the new microservice. The tool then can take that microservice’s response, hit the other remaining endpoints that still are in the monolith, conglomerate the response and send it back to the consumer as if it hadn’t changed.
Question: 
What convinced Uber to replace the monolith with microservices?
Answer: 
It was an engineering wide decision and it was about scalability in three ways. Scaling our software with the growth of the company; the monolith was not going to scale with a single, writeable Postgres database. It was not going to scale with the growth of our business.
The second way of scalability was feature scaling. Adding new features to the old monolith was slow. It was hard to test and any mistake would have a huge impact, and the impact could be unexpected. If we rolled out something into the monolith and it unexpectedly affected other parts of it, then it could take down all of Uber’s back-end.
The third way is scaling the organization. This huge codebase wasn’t owned by anyone. Many teams had their code in there, but supporting this was really tough because there are just so many teams committing to it and the ownership model wasn’t really there. Also, if new developers joining Uber were tasked with doing anything to this monolith, it would be daunting and could potentially be catastrophic if a mistake was made.
Question: 
What do you mean the ownership model wasn’t there?
Answer: 
There were many intertwined dependencies in this monolith, and it was hard to define who owns which section of code. Some things were clear. If the money team added a feature in the monolith, then they would own that feature. But if there were many dependencies on many different data models in the monolith that the money team was using, and if that team wanted to change their data model, then who was responsible for migrating that? There was just no clear ownership on the monolith as a whole.
Question: 
What are the key takeaways of your talk?
Answer: 
One of the things that I want to touch on is the fact that monoliths can have many benefits. There was a reason that we put all of our code in this one monolith starting five or so years ago, but at a certain point stability is more important than development speed, which was one of the benefits of having this monolith. You have access to all types of data in one place. When Uber was a lot smaller, we could roll out features a lot more quickly than if we had to integrate with a whole bunch of different microservices.
But as this became unstable and unscalable, we needed to break it up for those three scalability reasons that I mentioned. Then, through this migration, we have learned several things about how to approach a migration of this size. I want to share with the audience what we have learned so that if they undergo a migration- either from monoliths to microservices or any migration- what that takes and what we have learned about how to make it go smoothly.

Speaker: Emily Reinhold

Software Engineer @Uber

Emily is a software engineer on Uber's Money team. Since joining Uber in early 2015, Emily has been involved in many aspects of money, including charging riders and paying driver partners. She has recently contributed to the effort to dismantle Uber's monolith while building its microservice architecture. Before Uber, she studied computer science at UC Berkeley.

Find Emily Reinhold at

Similar Talks

Software Architect @VinSolutions, Author @pluralsight
Senior Director of Distribution Platforms @ESPN
Partner & Tech Lead @CarbonFive
Lead Data Scientist @betaworks
Leading Machine Learning Researcher, Vowpal Wabbit Contributor
Director - Penetration Testing / Red Team @Yahoo

Tracks

Monday, 13 June

Tuesday, 14 June

Wednesday, 15 June