Deploying ML Models in Production: An Overview

The deployment of ML models in production is a delicate process filled with challenges. You can deploy a model via a REST API, on an edge device, or as as an off-line unit used for batch processing. You can build the deployment pipeline from scratch, or use ML deployment frameworks.
In this video, you'll learn about the different strategies to deploy ML in production. I provide a short review of the main ML deployment tools on the market (TensorFlow Serving, MLFlow Model, Seldon Deploy, KServe from Kubeflow). I also present BentoM - the focus of this mini-series - describing its features in detail.
=================
1st The Sound of AI Hackathon (register here!):
musikalkemist.github.io/theso...
Join The Sound Of AI Slack community:
valeriovelardo.com/the-sound-...
Interested in hiring me as a consultant/freelancer?
valeriovelardo.com/
Connect with Valerio on Linkedin:
/ valeriovelardo
Follow Valerio on Facebook:
/ thesoundofai
Follow Valerio on Twitter:
/ musikalkemist​
=================
Content:
0:00 Intro
0:36 ML deployment strategies
1:32 Basic ML deployment
3:27 Disadvantages of basic ML deployment
4:57 Overview of ML deployment tools
9:54 BentoML
14:00 What's next?

[@Erosis]

ML Ops is such a rarely taught topic in academia. Thank you so much for this explanation. I hope there is more to come!

[@ValerioVelardoTheSoundofAI]

Indeed. Most of the time, academia is focused only on the research side. Productisation is mainly done in the industry.

[@fcojperez]

Good summary of all different ML "framework" (if we may agree on that) to deploy a ML model. Thank for that 🙏

[@Diego0wnz]

Almost starting as a data science consultant and I really need this video series!!! Comes at perfect time

[@ValerioVelardoTheSoundofAI]

Nice! Good luck with the new enterprise :)

[@stefanvasilev8948]

Thank you very much! This was the most helpful resource I've found on the topic!

[@travelwithak_]

Thanks very much for the detailed explanation. Really helpful👏

[@maxwohrle1049]

Great video as always. You really helped me a lot with my master thesis. This video series is yet another important building piece in dealing with audio machine learning projects. If I may add a comment. The Mnist dataset is a good example to demonstrate the deployment, but since the channel is called "the sound of ai" it would also be very interesting to see how to deploy a model that classifies an audio stream in real time.

[@ValerioVelardoTheSoundofAI]

Thank you Max. I will cover audio classification in realtime in a future series. I'd like to keep this mini-series is domain agnostic.

[@AbdurrahimKururu]

this is very helpful thanks

[@Coolblockj]

Hey! I've been trying to work on deploying some complex models and wondered if you had any experience in industry tools that do this well The scenario is essentially I have two models that need to be run, but one of the features they use is a completely separate model. Is there any industry standard software you know of to make that easier to do in a more scalable way? And if I eventually add more models that then depend potentially on those?

[@ahmedgoniboulama8423]

Hello, great video, well explained, i would like to know if BentoML can be used to Deploy MeshSegNet models?

[@arumugam2615]

Great ...

[@karndeepsingh]

Can you make a comparison video using ZenML and other MLOps tools?

[@sarthak7413]

Great Explanation..From where to get code files?

[@abdullrahmanelhayek2577]

Thank you so you amazing , videos , could you please make a vedio about how to use CHATGPT in programming

[@ValerioVelardoTheSoundofAI]

I recently created a voice assistant for ChatGPT. Check it out!

[@tirateable]

How are you? my question is different, please guide me I need to develop model language for Local language for speech recognation!

[@noahdrisort2005]

Currently I use torchserve

[@ValerioVelardoTheSoundofAI]

That's a good option!

[@sudharsankp]

I was really hoping if someone in the audio domain talks about deploying an speech model and give some pointers on ML Observability.

[@ValerioVelardoTheSoundofAI]

I won't be talking specifically about audio models in the next video. However, the same steps apply also in audio processing.