You are viewing content from a past/completed QCon -

Presentation: Programming for Hostile Environments

Track: 21st Century Languages

Location: Empire Complex, 7th fl.

Duration: 11:50am - 12:40pm

Day of week:

Slides: Download Slides

Level: Intermediate

Persona: Developer

What You’ll Learn

  • Hear use cases and tips on developing against globally distributed bare metal infrastructure.

  • Learn approaches that led Packet to move from a Ruby on Rail monolith to a Golang microservice architecture.

  • Learn concrete defensive programming techniques that can be used in environments that are often challenging.


In this presentation, we'll talk about some of the challenges that we've faced at Packet while attempting to program against the worst kind of adversary: physical, bare metal infrastructure. We'll describe a few real-world problems we've faced, general strategies to deal with environments that are not friendly to traditional developers, and practices that can be generalized to ensure good programming standards are maintained.


Your title is Programming in Hostile Environments. What does that mean?


At Packet, we deploy bare metal infrastructure. We deploy it globally across 16 locations.

When you're dealing with programs that have to operate on top of physical infrastructure, there are unfortunate realities that are not found in something like a virtual machine or a JVM runtime, and as a result we’ve needed to adjust how we develop our programs to account for these practical realities.

This talk will describe of the best practices and how  real-world situations that we've had to deal with internally at Packet.


Would you say this talk is a case study talk about Packet's road to programming in this environment or is it a best practice's story around things you need to do when you're building on bare metal?


I would say both.

We deploy bare metal on demand, so if you're a user of Packet you would expect a server to show up in between four and six minutes, and you would expect it to be successful. Almost as importantly, you expect to be able to do that 24 hours, a day seven days a week. Actually delivering on that promise is challenging because we're dealing with bare metal.

Bare metal systems are unreliable and may not show up when we expect them to show up. So I'll go through practical things that we needed to do to combat that reality as well as what that means for best practices around Golang.

I'll also discuss some of the choices that we made along the way as we transitioned from a Ruby on Rails based application into Go-lang microservices architecture, and how those services communicate with each other globally.


Can you give me an example of a pattern or defensive programming tip we can expect in your talk?


This is probably more of an anti-pattern. Imagine a pretty innocuous block of code where you're doing power and boot control on a system.

At Packet, we have front and center a block of code that says if the header that's being passed in indicates to us that there is a certain device manufacturer we're interacting with, perform certain logic based off of it. We've chosen this approach rather than exposing an interface and providing an object-oriented method of delivering the implementation because for us that hid from our engineers the reality that the code is going to be operating in an environment where you need to think about these things.

So we chose not to hide the code in an implementation of an interface. Of course, we do have interfaces and we do implement them for actual business logic in other places. But in this case, the choose helps keep where the code is operating forefront in the developer's mind.

Speaker: Nathan Goulding

SVP, Engineering @packethost

Nathan has been automating infrastructure for 15 years.

Find Nathan Goulding at

Similar Talks