Model deployment scenarios on Amazon SageMaker

[@caiyu538]

Find the answer from the question in your other lecture. Great.

[@muthutub]

Excellent video. Was very useful in grasping the different deployment options easily.

[@juliensimonfr]

Glad it was helpful!

[@studentlearner1453]

Hi Juien, would it be possible to make a video showing CI/CD pipeline for SageMaker, training the model in dev environment (AWS account) and promoting (hosting/inference) it on Prod environment using CodeCommit,CodePipeline,CodeDeploy etc. Jupyter notebooks to be version controlled perhaps in CodeCommit. Models to be version controlled perhaps in S3?

[@sanketg10]

Hi Aravind, have you figured out the CI/CD pipeline for SageMaker yet - just curious. Had the same questions.

[@studentlearner1453]

Hi Julien, great video and thanks for the responses on other questions. Would it be possible to also make videos demonstrating hands-on the 3 deployment scenarios? (I.e. if this video was theory, another video for hands-on)

[@anubhabjoardar9445]

Hi Julien, thanks for the video! Is it possible to have two models of different algorithms, say xgboost and blazing-text, in the multi-model endpoint scenario?

[@juliensimonfr]

not with the built-in containers. However, you *can* build a custom container, see docs.aws.amazon.com/sagemaker/latest/dg/multi-model-endpoints.html I guess it would be possible to load different model types, but I've never seen an example of that.

[@studentlearner1453]

If i need to have an endpoint that needs to be running 24x7 in production (inference), from a cost perspective, is it better to deploy an endpoint via SageMaker or is it better to use API Gateway-Lambda combination (i.e. lambda function only gets invoked when there is a request from a client, but otherwise we dont get billed). Thanks!

[@juliensimonfr]

If your model fits inside a Lambda package, and if it fits in the available memory, and if you can live with the occasional slow prediction caused by cold starts, then Lambda is a good option :)

[@caiyu538]

I will search your lecture how to implement it in code.

[@gloriamaciam]

awesome!!

[@juliensimonfr]

Thank you!

[@studentlearner1453]

4:54 "Create Models" is incredible confusing. I think it should instead be called "Register the model". Models would have already been "created" as part of the training job.

[@juliensimonfr]

I completely agree. This API is misnamed, I've been complaining since launch :)

[@MrJosutty]

Can I add any 2 models similarly? I want to add a custom algorithm and another one from sklearn. Is there a way I can do this if I modify the model_handler.py?

[@juliensimonfr]

Never tried it, but you should be able to do this with custom code, and then select the production variant that you want to invoke. Let me know :)