QCON NEW YORK 2017

Conference: Jun 26-28, 2017
Workshops: Jun 29-30, 2017

Presentation: Beyond Breakpoints: A Tour Of Dynamic Analysis

Track: 
Modern CS in the Real World

Location:

  • Salon E

Duration

Duration: 
5:25pm - 6:15pm

Day of week:

  • Tuesday

Level:

  • Advanced

Persona:

  • Developer

More talks on:

Key Takeaways

  • Understand the need and benefits of dynamic analysis for large scale distributed systems.
  • Hear about the dynamic analysis space and learn how to begin leveraging in your systems.
  • Learn to apply dynamic analysis to better understand the problems you are solving and how your code interacts with the data as it flow through your algorithms.

Abstract

Despite advances in software design and static analysis techniques, software remains incredibly complicated and difficult to reason about. Understanding highly-concurrent, kernel-level, and intentionally-obfuscated programs are among the problem domains that spawned the field of dynamic program analysis. More than mere debuggers, the challenge of dynamic analysis tools is to be able record, analyze, and replay execution without sacrificing performance. This talk will provide an introduction to the dynamic analysis research space and hopefully inspire you to consider integrating these techniques into your own internal tools.

Interview

Question: 
Can you tell me a bit about your role?
Answer: 
I work at a CDN called Fastly. I primarily have been working on the HTTP cache that serves our customers content to end users, spending most of my time dealing with performance problems in a highly concurrent multithreaded codebase. I am very interested in making improvements across all layers of the stack, so I have had to touch the kernel, application level code, SSL termination libraries and everything in between. A jack-of-all-trades low level systems person.
Question: 
What is the motivation for your talk?
Answer: 
In grad school, I spent a lot of time working on dynamic analysis tools; I spent a lot of time trying to figure how to make software more understandable for people. My thesis ended up being slightly different. It was about automatically improving a cache coherence problem using binary translation, which is a common dynamic analysis technique. But, it was less about providing visibility into software and more about making an opaque fix.
Functional programming is very hip right now, and there is a tendency for a lot of people to try and lean on static analysis tools, the type system or linters, to better understand what the program does. But, at scale and with respect to performance problems, it is hard to find a lot of that stuff statically. You have to introspect stuff at runtime to figure out what’s actually going on.
There seems to be a divide between what I learned in grad school with respect to dynamic analysis tools that academia is building and industrial practices for using dynamic analysis tools. So the hope is that this talk will encourage people to bridge into that gap and inspire them to start playing around with tools that already exist or encourage more forward thinking people to start building their own tools.
Question: 
Do you consider this an introductory or an intermediate talk?
Answer: 
The talk will make an introduction to the topic so in that sense, it could be geared towards beginners, but I expect us to get reasonably gory because we will be talking about how to solve concurrency issues, diagnosing kernel bugs by sticking instrumentation in virtual machine monitors like Xen. I feel like beginner programmers might find themselves underwater a bit quickly, so I would say intermediate.
Question: 
What are some of the key takeaways?
Answer: 
In the talk, we will be discussing a few dynamic analysis tools that are already out there. In particular, GDB's time-travelling debugging functionality, dtrace, Valgrind, maybe a couple of other open source tools. These are tools that people can use today. When you go home, if you are having trouble and you have a problem that you think this technique could be used for, you can go ahead and use them right away. That is the first level.
These tools are very extensible; they are almost frameworks for doing dynamic analysis. We think of Valgrind and as a memory checker but in actuality it's an entire platform that you can build all sorts of stuff in. So we will talk about limitations and tradeoffs and what you can build with these tools. That’s level 2, taking existing, off-the-shelf frameworks and building something new that the base package doesn’t let you do.
Then we will talk about some ideas from the research literature, such as building full system multicore binary analysis into a hypervisor. So this isn’t something that you can just plug in. You would have to construct the infrastructure to make this happen. But hopefully, people will be able to take away whatever level they feel comfortable with, being able to use this technique back home.
Question: 
Having come to your talk, what am I going to now be able to do?
Answer: 
Dynamic analysis is interposing introspection at different layers of the stack. There exists a wide range of dynamic analysis tools for the JVM that hook into JVM TI, for instance. That runs at the runtime level. It sits between the OS and the JVM. Conversely, for full system instrumentation, typically it is done at the hypervisor level so you subvert this idea that the OS is at the bottom of the stack and then you start observing the OS as it runs and conversely everything above it.
I like to encourage people to think beyond a dynamic analysis tool. What can you use as a shim to better understand what your program is doing or to modify what your program is doing? If you can peel underneath the layers and you try and wedge yourself in between them, you can observe stuff running under the hood or to modify it. That’s a worldview that is lacking, so hopefully, if we can change people’s perception on that space, that would be beneficial.
Certainly, the first level is letting you know you can use Valgrind on your software. But that is not necessarily super interesting. Many things that people in industry consider to be impossible are being actively worked on in the research world. People say that full system instrumentation is too hard and they leave it at that. And, they might conclude that there are no tools to do it because it is impossible to do. Whereas, people have been building this sort of stuff for a long time in academia; it just hasn’t made its way over to industry in the way I would hope that it would.
So, I hope the talk would encourage people to either at the first level pull stuff from Valgrind, or to use Valgrind or Pin to start building their own tools, or, even better, use the lit review to come up with new techniques in this space on their own.

Speaker: Nathan Taylor

Distributed System Engineer working on Caching @Fastly

Nathan Taylor is an Oakland-based software developer currently employed at Fastly , where he works on making the Web faster through high performance content delivery. Previous gigs have included hacking on low-level systems software such as Java runtimes at Twitter and, prior to that, the Xen virtual machine monitor in grad school. Originally a Trombone major, he holds an Msc. from UBC, where he researched full-system dynamic analysis and binary rewriting tools. When not in front of a computer, Nathan is likely to either be making granola or suffering up a steep hill on his road bike.

Find Nathan Taylor at

Speaker page

Cache Custodian at Fastly

Similar Talks

Learnings from a Culture First Startup
CTO @Buffer
Sunil Sadasivan
Becoming an Outlier
Software Architect @VinSolutions, Author @pluralsight
Cory House
ESPN Next Generation APIs Powering Web, Mobile, TV
Senior Director of Distribution Platforms @ESPN
Manny Pelarinos
The Human Side of Microservices
Tech Lead @Yelp
John Billings
The Seven (More) Deadly Sins of Microservices
Chief Scientist @OpenCredo
Daniel Bryant
Lessons Learned on Uber's Journey into Microservices
Software Engineer @Uber
Emily Reinhold
What They Don’t Tell You About Microservices…
CTO @Yodle
Daniel Rolnick
Algorithms for Animation
Partner & Tech Lead @CarbonFive
Courtney Hemphill
Machine Learning Fast and Slow
Lead Data Scientist @betaworks
Suman Deb Roy

Tracks

Monday, 13 June

Tuesday, 14 June

Wednesday, 15 June

See the Full Schedule

Conference for Professional Software Developers

Follow QCon

Contact

Menu

QCons around the World

Qcon