Episode Details
Back to Episodes
The Godfather Of MLOps // D. Sculley // MLOps Coffee Sessions #32
Description
Coffee Sessions #32 with D. Sculley of Google, The Godfather Of MLOps.
Get the newsletter: https://go.mlops.community/YTNewsletter
//Bio
D is currently a director in Google Brain, leading research teams working on robust, responsible, reliable, and efficient ML and AI. In his time at Google, D worked on nearly every aspect of machine learning and has led both product and research teams, including those on some of the most challenging business problems.
// Links to D. Sculley's Papers
ML Test Score: https://research.google/pubs/pub46555/
Machine Learning: The high-interest credit card of technical debt
https://research.google/pubs/pub43146/
Google Scholar:
https://scholar.google.com/citations?user=l_O64B8AAAAJ&hl=en
--------------- ✌️Connect With Us ✌️ -------------
Join our Slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with David on LinkedIn: https://www.linkedin.com/in/aponteanalytics/
Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/
Connect with D. Sculley on LinkedIn: https://www.linkedin.com/in/d-sculley-90467310/
Timestamps:
[00:00] Introduction to D. Sculley
[00:40] The Biggest Papers were written by D for Machine Learning
[02:08] What's changed since you wrote those papers?
[02:56] "No 1, there is an MLOps community."
[04:38] Old best practices
[05:12] "The fact that there are jobs titled MLOps, this is different than it was 5 or 6 years ago."
[06:30] Machine Learning Systems then and now
[07:08] "There wasn't the level of general infrastructure that was looking to offer the large-scale integrated solutions."
[07:57] ML Test Score
[11:09] "The Test Score was really written for situations where you don't care about one prediction. You care about millions or billions of predictions per day."
[12:27] "In the end, it's not about the score. It's about the process of asking the questions, making sure that each of the important questions that you're asking yourself, you have a good answer to."
[13:04] What else is needed in the Test Score?
[14:36] Stratified testing
[17:05] Counterfactual testing
[18:34] Boundaries
[19:15] Dark ages
[20:27] How do you try in Triage?
[21:10] "Reliability is important. There are no small mistakes. If there are errors, they're going to get spotted and publicized. They're going to impact users' lives. The bar is really high, and it's worth the effort to ensure strong relia