How to Deploy a Machine Learning Model to Google Cloud for 20% Software Engineers (CS329s tutorial)

It's time to reveal the magician's secrets behind deploying machine learning models! In this tutorial, I go through an example machine learning deployment scenario using Google Cloud and an image recognition app called Food Vision 🍔👁.
Get all the code on GitHub - github.com/mrdbourke/cs329s-m...
Slides - github.com/mrdbourke/cs329s-m...
Full CS329s syllabus - stanford-cs329s.github.io/ind...
Learn ML (my beginner-friendly ML course) - dbourke.link/mlcourse
Connect elsewhere:
Web - www.mrdbourke.com
Get email updates on my work - www.mrdbourke.com/newsletter
Timestamps:
0:00 - Intro/hello
1:42 - Presentation start (what we’re going to cover)
6:00 - Food Vision 🍔👁 (the app we’re building) recipe
11:16 - The end goal we’re working towards (data flywheel)
13:07 - The data flywheel: the holy grail of ML apps
14:57 - Tesla’s data flywheel
17:02 - Food Vision’s data flywheel
18:24 - Deploying a model on the cloud outline
21:14 - Steps we’re going to go through to deploy our app
27:06 - Question: “How do you identify hard samples in your data?”
37:53 - Creating a bucket on Google Storage
45:51 - Uploading to Google Storage from Google Colab
48:02 - Deploying a model to AI Platform
52:50 - Creating an AI Platform Prediction version
58:10 - Creating a Service Account to access our model on Google Cloud
1:02:32 - Authenticating our app with our private Service Account key
1:09:19 - What happens when we run make gcloud-deploy
1:11:27 - Problems you’ll run into when deploying your models
1:20:12 - Extensions you could perform on this tutorial
1:20:49 - Part 2 start (tutorial overtime)
1:28:43 - Dealing with different data shapes
1:32:35 - An error you might run into when using the example app (3 total models deployed)
1:33:20 - Dealing with data size restrictions
1:38:48 - Stepping through the make gcloud-deploy command
1:51:00 - Summary and wrap up
#machinelearning

[@mrdbourke]

**Update (20 May 2021):** After Google's recent I/O event, AI Platform has now changed its name to Vertex AI, so some of the terminology/steps on the GCP platform may be named differently/have slightly different steps, but the main principles of deploying a model should remain. Happy Valentine's Day team ❤️! Sending love to you all. Happy Machine Learning!

[@tirthpatel6436]

Can you please provide the required steps for VERTEX AI?

[@abhishek9721452503]

Hi Daniel ,I am trying to deploy the code on the Vertex AI,in this they have changed the format and specification of make_prediction() function which you used in video. Now when I am passing image as an instance to this new function I am getting error. Can you please update your code in the github for this entire deployment where you use the new VertexAI specifications?

[@sayakpaul3152]

Your honesty is infectious. The way you made your (Eng : ML) ratio clear at the beginning speaks volumes about it. Thanks, Daniel. Amazing work!

[@bhaveshrikame3888]

one of the best in the business thanks daniel for this wonderful tutorial, i always had problem about whole lifecycle of model on cloud, but you cleared all my doubts

[@ramonsantiago4573]

I would absolutely love for you to make a course about the software needed to deploy models. I'm having to learn bits and pieces from everywhere...

[@danielcasadoherraez4]

Incredible tutorial!! Thanks so much :D

[@yashpatel-wk7td]

The Content you provide in this Tutorials is my Valentine ❤️...!!! Thank you for posting top notch content..!!!

[@mrdbourke]

Woohoo! Happy Valentine’s Day Yash ❤️🌹

[@abhishekbhatia651]

You are such an inspiration Daniel!

[@yasminebaroud3965]

thank you, you are the best person for teaching every things

[@jorgefelipegaviriafierro705]

Thank you so much for this, really great content :)

[@tjv9qh]

I almost never comment on KZbin videos, but here I wanted to break that trend in order to kindly thank you for this tutorial. This was a MAJOR lifesaver for me in my SYS6016 (Machine Learning) course at UVA. Absolutely awesome stuff.

[@mrdbourke]

thank you! I appreciate your comment and you breaking your trend to leave it :)

[@SuperNova23333]

I just finished ZTM machine Learning course on UDemy. Thank you for all your hard work.

[@mrdbourke]

Massive effort Super! Congratulations on finishing!

[@timoose3960]

Great follow along tutorial! Keep it up!

[@mrdbourke]

Thank you Timoose! Plenty more to come

[@bob1234568790]

thanks for coming to our class!!! my favorite guest lecture so far!!

[@mrdbourke]

Thank you Lauren ❤️

[@ayushchakraborty4406]

Thank you for posting such amazing content, so Informative!!!!!😀😀

[@mrdbourke]

You’re welcome Ayush! Enjoy legend!

[@knowhrishi]

Thanks! That was awesome content! Which you always have! Love from India!

[@mrdbourke]

Thank you Hrishikesh! Love form Australia!

[@user-or7ji5hv8y]

Thanks for sharing the video with the community. very useful actually.

[@mrdbourke]

Thank you! Stoked you enjoyed

[@CortiPum]

Hi! First of all i found this super interesting. I´m doing something similar and this really helps me a lot. Ty !

[@mrdbourke]

Glad you enjoyed Tomás!

[@shoaibsh2872]

You read mind, I am sure about it. You have the power of telepathy. I was just searching about this topic an here you upload it right that moment. Thank you very much for the tutorial

[@mrdbourke]

Great timing hahaha! Enjoy Shoaib!

[@aritxatroyano2983]

Great video Daniel. Keep Going!!!

[@mrdbourke]

Thank you Aritxa! I appreciate it

[@cristophernimatuj4220]

Big fan of your work. I hope to get to your level one day.

[@mrdbourke]

Thank you Cristopher! All the best legend

[@yogeshhasabe4931]

A Great Gift for the people whose valentine is Machine Learning...!! Another Masterpiece from 'the Daniel Bourke' :)

[@mrdbourke]

Happy Valentines Day Yogesh! 🌹

[@phonethiriyadana]

wow.. this is great. was just looking for ML deployment resources 😊 could be a next step of your ML course.

[@mrdbourke]

Thank you! The models we used in this tutorial are built in the upcoming course

[@phonethiriyadana]

@@mrdbourke wo hoo.. can't wait :D

[@choc3732]

I had to do this for my college ML project last semester. Could really have done with this video....

[@mrdbourke]

I’m a little late!

[@pinakinchaudhari95]

Man this is just what I have started, wow

[@mrdbourke]

Enjoy Pinakin!

[@hemanthkotagiri8865]

In india, when you say "I'm a driver" some people look down on you. As apposed to that, I absolutely love the fact that you've mentioned you were #4.94 rated uber driver to the seemingly most intellectual minds of Stanford. That's being true of what you are. I like it. I'm learning a lot from you, Danny. Thank you!

[@mrdbourke]

Thank you Hemanth, I appreciate it! As for the “driver”, I treat the doorman with as much respect as the CEO. There’s opportunity to bring value to others no matter what your role.

[@hemanthkotagiri8865]

@@mrdbourke I totally agree with you on that. He/She is as human as I am and deserve respect.

[@mohamedabdelrehem7698]

i am bored on the train going home so i found your video best 2 hours of my life

[@mrdbourke]

Glad you’re enjoying it Mohamed!

[@Amandeep-kx8jl]

try to do videos on kaggle problems and contests. that would be very helpful

[@hamzakivideo]

Thank you sharing this! Quick question, how should I run tensorboard; I ran %tensorboard --logdir logs/scalar - but I keep getting an error of dataset not active

[@benxneo]

awesome video Daniel! In your opinion, should someone getting into data engineering learn GCP or AWS first?

[@mrdbourke]

Thank you Benedict! As for AWS vs GCP, I’d choose either, they both have similar offerings (just with different names for the services). AWS is probably more commonly used at different companies though

[@scientificthinking9257]

nice one

[@sonphamtrung1284]

That's great video. Thanks you. ---From Viet Nam with love---

[@mrdbourke]

Stoked you enjoyed! From Brisbane with love!

[@mkalime]

Hi Daniel and many thanks for the effort! Great video! Am I wrong in the following? (This is from the part where you use the app locally with the model deployed on GCP) When you created the model version, right at the bottom of the setup page (57:02), there was the option to choose a service account which you ended up not specifying at that point (neither later?). Then you created a service account, generated a key for it and saved it locally for the app to use when making a request. I did the same and managed to get it worked but only when I don't explicitly specify the sa. When I do specify the sa (during model version creation) I get "Failed to add IAM policy binding" error and model is not deployed. By playing around, I figured that the credentials for the specific sa don't even matter, GCP apparently has some default sa that it uses when one doesn't specify a custom sa.

[@mrdbourke]

Hmmm, I’m not 100% sure of what’s going on here. In my experience, I’ve had to make a SA for each specific service, or adjust an existing one for a new service - I thought GCP by default made all SA’s have as little permissions as possible, hence the requirement for creation. Perhaps I’m wrong or it’s been changed?

[@dcr7417]

Hi @Daniel Bourke thanks for this lesson! I'm looking forward to doing this project but am getting the following error: Collecting requests==2.25.1 Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB) ERROR: Could not find a version that satisfies the requirement tensorflow==2.4.1 (from versions: none) ERROR: No matching distribution found for tensorflow==2.4.1 Has anyone experienced this? My Python version in the virtual environment is 3.10.6

[@siddharthtiwari01]

Hi @Daniel Bourke, Thank you so much I learnt a lot. I noticed that the training code that you provided is not using GPU when I was training it myself on colab and on my PC . I wonder why? Any ideas?

[@mrdbourke]

You can switch to using GPU in Colab’s Runtime menu (when I trained the models I definitely used a GPU)

[@krumroar1572]

Nice ...go!

[@mrdbourke]

Thank you!

[@avinashreddyc8271]

Very good video , can i ask which clod platform is better gcp or aws or azure , gcp looks better in your tuitorial though

[@mrdbourke]

They all have similar offerings However, my experience is mostly with Google cloud So I went with them

[@pengdu7751]

great talk and demo! how does Google AI compare with other options such as AWS SageMaker?

[@mrdbourke]

Thank you! I haven’t compared Google AI and SageMaker but I imagine they’re similar

[@danielsilva3383]

Hello Daniel! I saw that you have a big interest in sports and even a degree. I am also passionate about sports and ML and if you don't mind I would like to ask some question. Do you think that there are good applications of ML in sports? Have you ever done anything related to that? How much expertise would it take? Do you think that building an app like SwingVision (tennis tracking app for iOS) is possible for one person? And what about sports analytics? Finally, could I make a Saas out of that? Have you ever done a Saas (of any type)? Thank you 😉☺️

[@mrdbourke]

SwingVision is definitely possible, check out something like “iOS pose detection” and let your imagination run I haven’t personally built a Saas but stay tuned Saas in sports would be same premise as other Saas though: find a problem people have, solve it with software, sell it

[@tarekhosny8166]

Daniel, Do I need to memorize the mathematical equations of machine learning algorithms or just understand the math behind them and what they do?

[@advait7802]

You need to just understand the math, no need to memorize them, because to do anything you'll be using tensorflow or any framework which will handle the math. You can actually skip the math (not recommended) and still be fine. The math is just so that you understand what's going under the hood. The math gets complex as you move ahead, so if you skip a bit of the end it's still fine as long as you understand all the basics. Although if you plan on doing deep research and figuring out new techniques that aid the current ML process, learning math is really really important.

[@tarekhosny8166]

@@advait7802 Thanks for the advice.

[@nicholai9266]

Is there any diff. between ML and AI engineers, does AI engineer(job role) really exist?

[@deekshithadp9545]

Hey Daniel, First of all thank you so so much for providing such informative content!! I sincerely appreciate all your efforts and hardwork. By watching this video, I have gathered plenty of information. I am going to continue watching all your videos in the coming days!! I am having few doubts. I will be greatful to you if you could guide me and suggest me your thoughts on this. 1. At 1:17:56 you mentioned about one class containing 11 classes while another class with random non-food images. What I understand from here is that you have got 2 main classes, one of them having 11 different classes, which makes it binary classification. How can I train this kind of image directory? I am having similar use case; I want to classify oral and non-oral images, where a non-oral class contaiins any random images of different categories. Could you please help me with this problem? How can classify my images? 2. Could you please let me what are the requirements/preparations that I need beforehand to deploy my trained model into GCP to use it in APP development? Thank you very much once again for such a valuable content!! ❤️ ❤️ ❤️ Keep rising and surprising people with more videos!!!!!

[@tsogoobayartsogt5653]

44:28 This is one way, but if you already connected with your Google Drive, then you'd be able to use GCP for that particular account (which I think a lot more easier)

[@tsogoobayartsogt5653]

Great content by the way

[@alpozen5347]

hey man im just curious, why did u not study CS since u seem to be doing it as a career now cuz i remember u mentioned u studied nutrition and sports

[@mrdbourke]

I didn’t even know CS was an option when I started university. But I’m glad I studied something else because now I’ve still got plenty to learn

[@kesavae9552]

3:41 I loved the way how he used kite image for small apps and plane for large apps 😂

[@mrdbourke]

🚀 soon

[@kesavae9552]

@@mrdbourke all the best

[@zeinfahrozi1733]

an error occured when trying to save model1 i think this is because using tf 2.7.0 KeyError: "Failed to add concrete function 'b'__inference_model_layer_call_fn_51753'' to object-based SavedModel as it captures tensor which is unsupported or not reachable from root. One reason could be that a stateful object or a variable that the function depends on is not assigned to an attribute of the serialized trackable object (see SaveTest.test_captures_unreachable_variable)."

[@fiqiamali]

Hey daniel, when I run model 1,2,3 on google collab, why it cannot be save like you did. Mine tell there is an error

[@mrdbourke]

Yo yo, what error are you getting? Any chance you could post it on the Github for the tutorial (as an issue)?

[@RisuAnetto]

How can I use Pytorch model? Do I need a custom container? What container is useful for a BERT model?

[@mrdbourke]

You can definitely use PyTorch but it requires a couple different steps, check the documentation on AI Platform for more

[@buffetbar8231]

How to decide the number of hidden layers and nodes in a hidden layer?

[@shreyasingla9307]

👍🏻👍🏻

[@crehenge2386]

So if I do something, I will have done it... wow, that's deep

[@reazonsraj]

Dan mate are u sing the apple m1 or just a regular mac cus my m1 is having issues installing all this shit

[@mrdbourke]

I used a non-M1 Mac for this video (MacBook Pro 16-inch, i9)

[@reazonsraj]

@@mrdbourkei can't install all this on my m1 so switched to windows btw loved the zero to mastery on tensorflow any way u can do a tutorial on deployment a little bit more detailed than this cs class plzz mate we are asking for it for a while after we bought the course cheers mate.

[@alexchan5643]

47:20 my exact reaction whenever anything goes right

[@mrdbourke]

Hahaha I wanted to make it as real as possible

[@prod.kashkari3075]

Is pytorch equally as good for deployment as tensorflow is?

[@mrdbourke]

Yes, however I think currently TensorFlow edges it out with features (eg mobile deployment)

[@alikafaei102]

Love the aussie accent bo

[@shubhammuramkar9316]

make gcloud-deploy is not working timestam 1:09:32. says "zsh: command not found: makefile"

[@dhruvmewada4303]

shuold i switch to linux ubantu for machinelearning or windows will work fine'

[@mrdbourke]

Windows works fine, however, most cloud systems run some form of Linux (Ubuntu etc)

[@nishchaysingh5980]

Excuse me Sir. I know that you have earned the Tensor Flow Developer Certificate . Congratulations on that. I am also following your instructions that you gave about that Course Course Of Tensor Flow. Unfortunately before that I referred to random video on KZbin and tried to use some basic commands of Tensor Flow on Jupyter notebook but a lot of problem start coming with the version . Mostly about ".... . some command could not be found in module" . Also tried to downgrade the version to 1.4 but not much happened. It had became a headache . Can you give some help so that these things do not restrict me from implementing commands while doing the Coursera Course Of Tensor flow. 🙏🙏🙏🙏

[@mrdbourke]

If you’re running into troubles with Coursera’s notebooks, I’d use Colab and practice writing as much TensorFlow code as possible, you will learn it

[@kolla_teja]

I would recommend don't go for lower version check once on tensorflow official certification examination page. you can get to know which version you are really want to use and on which version your exam based. happy learning!

[@talhakhursheed2467]

How much will it cost me to deploy my ML model on GCP?

[@002_aarishalam8]

converting floating points to int16 is giving weird results.

[@ameenbello4500]

Does anyone have the link to his twitch channel, can't seem to fnd it anywhere

[@mrdbourke]

twitch.tv/mrdbourke

[@markgacoka9704]

While other people are on Valentine's, I'm busy trying to deploy ml models 😂😂

[@mrdbourke]

You and me both! 🌹+💻

[@MrBemnet1]

can you shrink the video length for the future? It is too long ..

[@programmerrdai]

How can I join the course?

[@mrdbourke]

Full details on the course here: stanford-cs329s.github.io/

[@jithendrayenugula7137]

You using Google instead of duckduckgo? 😐 It's from your videos I started using duckduckgo 😅 Anyway great tutorial...Thanks for making it public🤟

[@mrdbourke]

I use both! And thank you for the kind words

[@danielsilva3383]

Hello, Great Video! What do you do for a living? Does this channel pays for your bills? Could you make a video on how much you earn from YT and your courses, please? Thank you

[@mrdbourke]

Hey Daniel, thank you for the kind words. This article sheds a bit more on how I make money: www.mrdbourke.com/running-a-business-from-a-bedroom-11-months-in/

[@danielsilva3383]

@@mrdbourke Thank you

[@smerlinger]

How old are you?? Curious to measure your age and my age and where i'm at in my career!

[@mrdbourke]

I’m 27

[@davedrives212]

PS1 Malfoy never ages

[@DiegoRivera-eu4uh]

Chicken curry?!! haha

[@jackbauer322]

Not pizza :)

[@mrdbourke]

Pizza, not pizza!

[@user-zo4mu6bp6w]

Hey Man! You streamed such a great tutorial for free? Thank you Sir for your community services!