Session + Live Q&A
Streaming-first Infrastructure for Real-time ML
Because of data drift, the accuracy of an ML model degrades over time. How well a model performs depends on how often it’s updated. While companies like Alibaba, ByteDance, Google, Facebook have been able to leverage real-time pipelines to continually update many of their models in production and boost their performance, many companies still update their models manually. In this talk, we’ll discuss the state of continual learning for ML, its motivations, challenges, and possible solutions.
Main Takeaways
1 Find out how a streaming-first infrastructure can help you do ML in real-time, both online prediction and continual learning.
2 Discover why real-time ML is getting traction in the industry.
3 Learn what are some of the challenges to implement real-time ML.
What is the focus of your work these days?
My work has been in helping people bring ML into productions. I come from a research background, then I joined NVidia in their applied research team, where I realized a lot of the challenges are in production. It's not that the research is not interesting. I think research is extremely important. But I don't think I have the best skillset for research, whereas I think I can make my impact in production. Currently, I'm working on a startup. We are still in stealth. Our goal is to help companies leverage streaming-first architecture to do real-time machine learning, both for online prediction and continual learning. That's going to be the topic of my talk as well. And the nice thing about streaming infrastructure is that you get a good view of the data transformation through the entire system. So it's really good for model observability. I'm also teaching machine learning systems design at Stanford where I feel like I learn more from students than they learn from me.
How would you describe the persona and the level of the target audience?
Definitely is a technical talk, so it's geared toward both machinery engineers who develop ML models and DevOps engineers who have to deploy and maintain ML models. I'm a huge believer there's a big difference between things that are inherently complex and things that people just make complex because they can't find an easy way to explain. So even though the talk is technical, I hope to make it easier to understand for a less technical audience. We’ll be briefly discussing real-time ML from the business perspective as well. I think the problem with adapting real-time machine learning is that it requires a non-trivial infrastructure investment, so you need buy-in from leadership. I hope that there will be some technologically-progressive business leaders who watch my talk and say, it is an interesting direction and maybe try to experiment to see if it works for the organization.
What would be some of the key takeaways they would get?
I hope that they would have a better understanding of what real-time machine learning means and the challenges around deploying it, adopting the infrastructure for real-time machine learning. Also, I would like to show some of the interesting use cases and motivations for why real-time machine learning is gaining traction in the industry. Some of the answers that I hope they would find interesting to think about, for example, how to evaluate whether ML is right for your use cases.
Speaker
Chip Huyen
Founder at stealth startup & Teaching ML Sys @Stanford
Chip Huyen is an engineer and founder working to develop tools for ML models to continually learn in production. Through her work with Snorkel AI, NVIDIA, and Netflix, she has helped some of the world’s largest organizations deploy machine learning systems. She teaches Machine Learning...
Read moreFind Chip Huyen at:
Date
Tuesday Nov 9 / 01:10PM EST (40 minutes)
Track
Topics
Machine LearningData EngineeringArtificial IntelligenceAdd to Calendar
Add to calendarShare
From the same track
What You Should Know Before Deploying ML in Production
Tuesday Nov 9 / 11:10AM EST
MLOps provides tools that make building, deploying, and maintaining machine learning solutions easier than ever before. However, MLOps is not only a static set of tools that defines the way you operationalize your machine learning models, but is also (and most importantly) about your organization...
Francesca Lazzeri
Principal Data Scientist Manager @Microsoft
The Unreasonable Effectiveness of Zero Shot Learning
Tuesday Nov 9 / 12:10PM EST
A long and expensive phase of machine learning projects is collecting and labelling data, and training ML models. In this talk I will show how one can get started deploying models without requiring any data. We talk about what foundational models are, and go through examples of them, such as...
Roland Meertens
Product Manager @annotell
ML Panel: "ML in Production - What's Next?"
Tuesday Nov 9 / 02:10PM EST
The panel will discuss the current lessons learned with putting ML systems into production.What is working and what is not working, from building ML teams, dealing with large datasets, governance and ethics/privacy issues, and what's around the corner for production ML, and ML in computing...
Chip Huyen
Founder at stealth startup & Teaching ML Sys @Stanford
Shijing Fang
Principal Data Scientist @Microsoft
Vernon Germano
Senior Manager of Machine Learning Engineering @zillow