Qconn

How Facebook Scales Big Data Systems

How Facebook Scales Big Data Systems

Time: 
Wednesday, 5:30pm - 6:20pm
Abstract: 

Facebook uses distributed storage systems that individually have tens of thousands of servers across multiple data centers around the world, receiving millions to billions of queries per second. At this scale, it is essential to find a malleable storage solution that is fast, easy for both engineers and operations staff to use, and provides strong safety guarantees.

 

Jeff Johnson, Software Engineer at Facebook New York, will share -- for the first time -- learnings on a new Facebook NoSQL project. The project's aim is to create a storage system simple in concept, implementation and operation, that offers a solution to the breadth of popular systems lacking one or more of these foundational elements. In comparison to many current NoSQL systems, this new system is based around using strong consistency as a core building block, implemented with the Raft consensus protocol. Jeff will detail how this approach is used to tackle storage problems at Facebook scale, and how issues of data availability and placement are handled.

Jeff.Jonhson's picture
As a Software Engineer at Facebook New York, Jeff leverages his expertise in large-scale data replication, distributed computing and coordination, data storage, and a range of other engineering skills to improve the overall Facebook experience for its 1.23 billion users. Prior to Facebook, Jeff spent over a decade of work in the video games industry, where he developed both client and server systems for Lord of the Rings Online and Dungeons & Dragons Online. Jeff graduated from Princeton University with a degree in Computer Science.