Presentation: From Software Development to ML - A Team's Transformation
Share this on:
What You’ll Learn
- Hear experience and challenges leading a traditional software engineering team into a machine learning/AI journey.
- Hear recommendations on how to approach the idea of transitioning expert-level developers into a new field.
- Understand what the priorities are when teaching a team new skill in machine learning.
Abstract
As companies seek to "inject" Machine Learning into their existing operations, they inevitably face a number of challenges. But perhaps the greatest and most-overlooked one is the people challenge. How can a successful software engineer make a transition to become an equally capable Machine Learning or Data Engineer?
How do team dynamics change when ML-specific processes, tools and pipelines are introduced? How can an engineering manager or team lead navigate this transition to retain their talented engineers and build a cohesive, high-performing ML team?
In this talk I will share some "war stories" and lessons learned from my own experience leading a multi-functional team through such a transition. The insights will hopefully be useful to both engineers and managers about to go into this experience.
QCon: What is the focus of your work?
Sally: Almost a year ago, I quit my job to finish my Phd. in AI and BioInformatics. It involves the use of AI techniques in the research of genetics and computational biology. Based on my Phd. research, I am also starting a company in the same space.
QCon: Your talk is called “From Software development to ML: A Team’s transformation”. What is the motivation for the talk?
Sally: The motivation came from my own experience leading a traditional software team into a machine learning. I had previously led a team for a health tech startup which was focused on AI and ML.
Before I joined this company, the team had been working on traditional software development projects in PHP. I was required to transform this team to an AI team to implement a challenging product for predicting health care needs. However, the biggest challenge turned out to be, turning a traditional software team into a high performing AI team.
Since many companies are trying to achieve this transformation, I wanted to share my experience.
QCon: What exactly does this transition to an AI-ML team comprise of?
Sally: The team had to learn Python from scratch. I had to hire some additional Python developers. Going from being a superstar PHP developer to a Python beginner was a challenge for some of the team members.
The mindset of a Python engineer is very different. It requires understanding a different framework and the Machine Learning pipeline. The individual also needs to understand how to deal with high volumes of data coming in at high speeds from different sources.
There are differences from a team perspective as well. For example, the Data Scientist role within a team is new for a traditional development team. AI-ML teams need to interact more with DevOps teams.
The challenge for me was not only to acquire a new skill set but also to hold the team together during this transition and maximize everyone's potential and strengths.
QCon: What will be the focus of your talk? Culture, tools, and/or people?
Sally: Culture and people will be my primary focus while tools and processes will be secondary. The main focus will be on how to deal with individual team members going through this transition. I will share a few examples of people and situations from my experience. I will share the lessons I learned from mistakes as well as the decisions which worked well for my team.
I will also talk about how the others in the organization perceive the AI-ML team and what are their expectations.
QCon: Who is the primary audience this talk is intended for?
Sally: Primarily, it is for team leaders who could be also be engineering managers or project managers. It should also be relevant to senior developers or engineers who are part of a team that is being transformed into an ML team.
QCon: What tips and recommendations would you give to an individual developer who wants to adopt AI and ML in their career?
Sally: The individual should be willing to invest a lot of personal time in learning. Self Learning and Self Motivation are important. I would recommend going through online courses and participating in competitions on Kaggle or looking at what others have done on GitHub.
You also need to make your supervisors aware of your interest, so that they can help you.
QCon: What do you think is the most important trend in software today?
Sally: I'm obviously biased, so I'll talk about AI. I am really excited to see what happens with the combination of AI and voice applications in the next few years. I believe voice will be the new main interface to many personal devices in the near future, and I don't think we've quite cracked it yet both on the voice recognition and generation fronts. In my opinion, voice interfaces combined with NLP techniques, deep learning and IOT will be the next wave.
Similar Talks
Tracks
-
Microservices: Patterns & Practices
Evolving, observing, persisting, and building modern microservices
-
Developer Experience: Level up Your Engineering Effectiveness
Improving the end to end developer experience - design, dev, test, deploy, operate/understand. Tools, techniques, and trends.
-
Modern Java Reloaded
Modern, Modular, fast, and effective Java. Pushing the boundaries of JDK 9 and beyond.
-
Modern User Interfaces: Screens and Beyond
Zero UI, voice, mobile: Interfaces pushing the boundary of what we consider to be the interface
-
Practical Machine Learning
Applied machine learning lessons for SWEs, including tech around TensorFlow, TPUs, Keras, Caffe, & more
-
Ethics in Computing
Inclusive technology, Ethics and politics of technology. Considering bias. Societal relationship with tech. Also the privacy problems we have today (e.g., GDPR, right to be forgotten)
-
Architectures You've Always Wondered About
Next-gen architectures from the most admired companies in software, such as Netflix, Google, Facebook, Twitter, Goldman Sachs
-
Modern CS in the Real World
Thoughts pushing software forward, including consensus, CRDT's, formal methods, & probalistic programming
-
Container and Orchestration Platforms in Action
Runtime containers, libraries, and services that power microservices
-
Finding the Serverless Sweetspot
Stories about the pains and gains from migrating to Serverless.
-
Chaos, Complexity, and Resilience
Lessons building resilient systems and the war stories that drove their adoption
-
Real World Security
Practical lessons building, maintaining, and deploying secure systems
-
Blockchain Enabled
Exploring Smart contracts, oracles, sidechains, and what can/cannot be done with blockchain today.
-
21st Century Languages
Lessons learned from languages like Rust, Go-lang, Swift, Kotlin, and more.
-
Empowered Teams
Safely running inclusive teams that are autonomous and self-correcting