QCON NEW YORK 2017

Conference: Jun 26-28, 2017
Workshops: Jun 29-30, 2017

Presentation: Safe Systems Programming in C# and .NET

Track: 
Incredible Power of an Open-Sourced .NET

Location:

  • Dumbo / Navy Yard

Duration

Duration: 
11:50am - 12:40pm

Day of week:

  • Monday

Level:

  • Intermediate

Persona:

  • Developer
  • Developer, .NET

More talks on:

Key Takeaways

  • Learn how you can use C# for systems level programming.
  • Hear patterns and practices for Systems level development.
  • Push existing C# skills into a new realm or learn a possible use of .NET in a way you may not have considered previously

Abstract

Systems software must be written in C, with unsafe pointers, manual memory management, and the associated cuts and bruises. Or so common wisdom says. This isn't as true as it once was, thanks to the confluence of many trends on both client and server: mobile, constant connectivity, and new trends in isolation like containers. The truth is, with the right discipline, you can reap the productivity, safety, and reliability benefits of using a managed language like C#, while still having great performance and scale. In this talk, I will share some of my key experiences from building an entire operating system in a C# dialect, with a focus on areas like garbage collection, low-level code quality, and dealing with errors and concurrency robustly. Although the patterns and lessons transcend any one language, examples will focus specifically on using the open source C# and .NET projects as they exist today.

Interview

Question: 
What’s your main role today?
Answer: 
I am Director of Engineering for the languages and compiler teams at Microsoft. I manage a group of about 200 engineers building the languages, the compilers, the tools, the IDE support, the static analysis tools. In addition to managing, I help set the technical strategy and technical direction for the group.
Question: 
Obviously, you are talking systems programming, but what is the main motivation for this talk?
Answer: 
When we started the Midori project, that was an experiment to see “Hey, could you write an entire operating system in a language that is safer than C and C++, and could the result compete performance wise with commercial systems like Windows and Linux?” What would be the challenges that you would run into? Would it really deliver on the promises, more reliability, better security?
As an industry, we still face a steady stream of security exploits. Just yesterday, there was another OpenSSL one, some image processing library one, and even more the week before that. Of course, most of these components are written in C. It’s not that C is a bad language. C is a great language. I love C. I’ve loved writing C code for a long time. It’s the manual memory management, the associated pitfalls, and some other more subtle things like arithmetic overflows, incorrect error handling, and so on, that you see in these exploits that are coming out. I believe security is the most important problem for the software industry to focus on at the moment. A safe language starts from a stronger foundation where many of these problems are eliminated “by construction.”
The question is, however, how can you can take a safer language and make it appropriate for systems programming?
Many lessons that I learned through the Midori project, and subsequently tried to apply to Windows, .NET, and C++, led me to that, once you have safety, much of the remainder boils down to patterns and best practices. Like don’t allocate gobs of memory when there’s a better way. Of course, having a great language, type system, and runtime helps immensely, and you see this in some new languages like Rust, but you can go a long way with the tools in your hands today.
It turns out, concurrency and distributed programming have fundamentally changed systems programming. The sorts of performance concerns that dominated when C was the only serious choice for systems programming actually apply less now, or at least have changed to the point where you can actually be successful with a language like C#, Java, Go, Rust, some of these “higher level” languages.
Question: 
Is the talk focused on patterns and practices? Is it focused on a language?
Answer: 
I want the talk to appeal to existing C# and .NET developers and will talk about the problems you often see. To foreshadow a bit, a lot of them have to do with garbage collection.
But there are others too around overflows, exception handling, reliability.
I want to talk about the problems and how to solve them in C#. Here is why something is problematic in practice, and how might you attack it. Here is what we did in Midori and then, here are some things that are coming in C# 7. In some cases, we actually took features straight from Midori and are adding them to C# 7 and the next version of .NET.
But again, a lot of it does come down to practices, and the framework, it turns out, is as important as the language. We have a separate repo alongside the main .NET one, https://github.com/dotnet/corefxlab, that contains some experimental abstractions that you can use in your code today, using the existing compilers, that can make a big difference. I also have some static analyzers and new types that I’ll open source, to help people do the right thing, like avoiding allocations where they don’t want them.
Question: 
Is your talk about what you can do today for systems programming? Or is it about the future?
Answer: 
I want this talk to be relevant to people writing code today. For every problem I will present a solution that you can implement today.
If you want to do systems programming, you really need to know what you’re doing. You are going to have to go out of your way a little bit to be successful. But if you are willing to go that extra step today, then sure, you can be successful with today’s technology.
For example, Span. We have a prototype of Span in the corefxlab repo and are experimenting with incorporating it deep into the framework and the language itself. Span isn’t magical, and in fact, was written using existing .NET constructs. Remember, C# has unsafe code which, though it is the root of all of C’s problems, can be encapsulated to build safe and efficient abstractions, even in C#.
In summary, yes, you can write efficient systems programs in C# today, if you are willing to take the time. But you are less likely to fall into the pit of success without seriously knowing what you’re doing. You’ll probably need to know a little about the insides of how a garbage collector works, for example. However, my teams are pursuing making all of this much easier to do the right thing “by default” down the road.
Question: 
Why would a JVM or Go developer that is at QCon come at this talk?
Answer: 
The focus on patterns and things that you will see in the wild is interesting. Go is a garbage collected language and, although Go has slices, there are plenty of other problems that I still see and I regularly encounter in my own Go programs as well. Java has many overlapping challenges too, though like Go it has addressed one of the challenges in an interestingly controversial way. People might be surprised to hear my take on checked exceptions since we had a variant of this in Midori.
C++ is also super interesting. To be frank, the Midori project has more rapidly made a mark on the shape of C++ than it has C#! After coming to the developer division, I took a lot of lessons learned from the Midori project, and we have been socializing those ideas with the C++ standards body, and launched a new effort to codify best practices in partnership with folks like Bjarne Stroustrup. You can read those on GitHub at https://github.com/isocpp/CppCoreGuidelines.
I see this whole thing as a big community of systems programming. We need to all learn together as a community and band together to fundamentally change the way software is written. I really do believe the security situation is the biggest problem for the industry right now. With IoT, cloud, and AI continuing to explode, the current state is frankly frightening. Instead of tackling the problem in private silos, we need to instead be open about our approaches, talk to each other, learn from one another, and help each other.

Speaker: Joe Duffy

Leads the Teams Building Languages/Compilers for C++, C#, VB, & F# @Microsoft

Joe Duffy is Director of Engineering for the Compiler and Language Group at Microsoft. In this role, he leads the teams building C++, C#, VB, and F# languages, compilers, and static analysis platforms, across many architectures and platforms. His group began, and is now responsible for elements of, taking .NET cross-platform and open-source. Prior to this, Joe was an architect on the Midori OS and led the creation of .NET's Task<T> and overall concurrency programming models. He has 2 books, 80 patents, and loves all things code.

Find Joe Duffy at

Speaker page

http://joeduffyblog.com/
Partner Director of Engineering at Microsoft

Similar Talks

Learnings from a Culture First Startup
CTO @Buffer
Sunil Sadasivan
Becoming an Outlier
Software Architect @VinSolutions, Author @pluralsight
Cory House
ESPN Next Generation APIs Powering Web, Mobile, TV
Senior Director of Distribution Platforms @ESPN
Manny Pelarinos
The Human Side of Microservices
Tech Lead @Yelp
John Billings
Lessons Learned on Uber's Journey into Microservices
Software Engineer @Uber
Emily Reinhold
What They Don’t Tell You About Microservices…
CTO @Yodle
Daniel Rolnick
Algorithms for Animation
Partner & Tech Lead @CarbonFive
Courtney Hemphill
Machine Learning Fast and Slow
Lead Data Scientist @betaworks
Suman Deb Roy
Vowpal Wabbit a Machine Learning System
Leading Machine Learning Researcher, Vowpal Wabbit Contributor
John Langford

Tracks

Monday, 13 June

Tuesday, 14 June

Wednesday, 15 June

See the Full Schedule

Conference for Professional Software Developers

Follow QCon

Contact

Menu

QCons around the World

Qcon