Panel: Navigating the Future: LLM in Production

Our panel is a conversation that aim to explore the practical and operational challenges of implementing LLMs in production. Each of our panelists will share their experiences and insights within their respective organizations.

The panel will provide a view of the current state of LLMs in production environments. Our goal is to stimulate thoughtful conversation and exchanges of ideas among AI researchers, software engineers and tech leaders. We strive to foster a nuanced understanding of the current landscape of LLMs in production and anticipate its future directions.


Sherwin Wu

Technical Staff @OpenAI

Sherwin is a Member of Technical Staff at OpenAI. He works on the Developer Platform team, which is responsible for the products that allow developers to build products on top of OpenAI models and capabilities.

Read more


Hien Luu

Sr. Engineering Manager @DoorDash

Hien Luu is a Sr. Engineering Manager at DoorDash, leading the Machine Learning Platform team. He is particularly passionate about the intersection between Big Data and Artificial Intelligence. He is the author of the Beginning Apache Spark 3 book. He has given presentations at various conferences such as Data+AI Summit, XAI 21 Summit, MLOps World, YOW Data!, appy(), QCon (SF,NY, London).

Read more


Rishab Ramanathan

Co-founder & CTO @Openlayer

Rishab is the co-founder & CTO of Openlayer. Openlayer is a YC-backed company that aims to make it easy for ML teams to test their models, diagnose underlying failure patterns, and take corrective action. Think Postman for ML. Rishab graduated from Yale in 2019, and subsequently worked at Apple on a variety of projects within their AI/ML org before founding Openlayer.

Read more


Wednesday Jun 14 / 05:25PM EDT ( 50 minutes )


Salon D


Video is not available


Slides are not available


From the same track

Session Machine Learning

Improve Feature Freshness in Large Scale ML Data Processing

Wednesday Jun 14 / 11:50AM EDT

In many ML use cases, model performance is highly dependent on the quality of the features they are trained and inference on. One of the important dimensions of feature quality is the freshness of the data.

Speaker image - Zhongliang Liang
Zhongliang Liang

Engineering Manager @Facebook AI Infra

Session MLOps

Platform and Features MLEs, a Scalable and Product-Centric Approach for High Performing Data Products

Wednesday Jun 14 / 04:10PM EDT

In this talk, we would go through the lessons learnt in the last couple of years around organising a Data Science Team and the Machine Learning Engineering efforts at Bumble Inc.

Speaker image - Massimo Belloni
Massimo Belloni

Data Science Manager @Bumble

Session AI/ML

A Bicycle for the (AI) Mind: GPT-4 + Tools

Wednesday Jun 14 / 02:55PM EDT

OpenAI recently introduced GPT-3.5 Turbo and GPT-4, the latest in its series of language models that also power ChatGPT.

Speaker image - Sherwin Wu
Sherwin Wu

Technical Staff @OpenAI

Speaker image - Atty Eleti
Atty Eleti

Software Engineer @OpenAI

Session ML Infrastructure

Introducing the Hendrix ML Platform: An Evolution of Spotify’s ML Infrastructure

Wednesday Jun 14 / 10:35AM EDT

The rapid advancement of artificial intelligence and machine learning technology has led to exponential growth in the open-source ML ecosystem.

Speaker image - Divita Vohra
Divita Vohra

Senior Product Manager @Spotify

Speaker image - Mike Seid
Mike Seid

Tech Lead for the ML Platform @Spotify


Unconference: MLOps

Wednesday Jun 14 / 01:40PM EDT

What is an unconference? An unconference is a participant-driven meeting. Attendees come together, bringing their challenges and relying on the experience and know-how of their peers for solutions.