Creating APIs For Machine Learning Models with FastAPI

Рет қаралды 11,912

Күн бұрын

Today we learn how to easily turn machine learning models into usable APIs using FastAPI in Python.
◾◾◾◾◾◾◾◾◾◾◾◾◾◾◾◾◾
📚 Programming Books & Merch 📚
🐍 The Python Bible Book: www.neuralnine.com/books/
💻 The Algorithm Bible Book: www.neuralnine.com/books/
👕 Programming Merch: www.neuralnine.com/shop
💼 Services 💼
💻 Freelancing & Tutoring: www.neuralnine.com/services
🌐 Social Media & Contact 🌐
📱 Website: www.neuralnine.com/
📷 Instagram: / neuralnine
🐦 Twitter: / neuralnine
🤵 LinkedIn: / neuralnine
📁 GitHub: github.com/NeuralNine
🎙 Discord: / discord

Пікірлер: 23

@omegasigma4500 Ай бұрын

I'm glad you uploaded a video about FastAPI. We prefer it over Flask. There are 2 topics where we need some help. 1.) Hosting - How to deploy the app so that other can access it via web? And how to manage the cloud infrastructure? 2.) Frontend - There are now plenty of frameworks and libraries. The standard approach is probably JavaScript, HTML and CSS. But I'm wondering what you think about pure Python libraries like Taipy, FastUI and reflex. What do you think is the best approach here? We would highly appreciate your input. Thanks! Keep up the great work! 💪💪👍👍

@khandoor7228 Ай бұрын

This was excellent, the capabilities this opens up is really powerful. Good job as always.

@TheDigitalSight Ай бұрын

We use FastAPI more than django and flask, can you please create video on langchain and fastapi as well?

@user-lg6dl7gr9e Ай бұрын

@cheukmingau983 Ай бұрын

In production the async endpoint should not be used. An async function (coroutine) will be executed in the main thread event loop, and like the event loop in JS inside the browser, it can only execute one coroutine at a time. Running the synchronous, cpu intensive `model.predict` inside the async endpoint will make your prediction endpoint frozen and wait for the underlying cpu predicting the images, so the QPS of your handler is at most one. Better options could be: 1) Using a synchronous function as the inference endpoint, 2) create a threadpoolexecutor outside of the async function, and use `loop.run_in_executor()` with the threadpoolexecutor declared as it will run the model inside the thread, or 3) use poolexecutor similar to option 2. The problem for option 3 is that multiprocesses requires pickling and you might have to tweak your model case by case. Also, pickling the model and deserialize in the application api server doesn't reveal the identity and method signatures of that model. If you are the only one who train and deploy that might not be a big problem, but in production you might want to use some inferencing frameworks like Onnxruntime which you just serialize your trained model to the preferred format first (onnxruntime has a very small package size compared to other DL libraries which makes the deployment dependency smaller). Lastly, running scikit-learn model in python doesn't utilize the all the cores in your cpu, whereas other packages usually have higher utilization proportion. I understand that the model in this video is small in size and is a POC, so with the small size running async and pickling is fine. However, for just some even better CV and NLP models (e.g. BERT) it is nearly impossible to adopt the same approach as in this tutorial.

@alexandrosmaragkakis737 Ай бұрын

@cheukmingau983 hello, I'm currently facing a problem very similar to what you described, is there somewhere I can message you to get more info on this?

@cheukmingau983 Ай бұрын

@@alexandrosmaragkakis737 perhaps here? You can state your situation just with the minimal details

@timothyelems1357 Ай бұрын

Exactly what I was looking for! Thanks man!

@dipeshsamrawat7957 Ай бұрын

You are making requested videos. Thank you 💯

@thelifehackerpro9943 Ай бұрын

why not use model directly instead of pickle?

@tthcan8038 Ай бұрын

Pickle makes your model remember the weights, so you need to fit only once

@smstudio1035 Ай бұрын

Can we see a hosting video off the same

@zedcodes Ай бұрын

Why do I have this error: 'module 'PIL.Image' has no attribute 'ANTIALIAS''? @10:41

@bobfreeman7349 Ай бұрын

ANTIALIAS was removed in Pillow 10.0.0 (after being deprecated through many previous versions). Now you need to use PIL.Image.LANCZOS or PIL.Image.Resampling.LANCZOS. (This is the exact same algorithm that ANTIALIAS referred to, you just can no longer access it through the name ANTIALIAS.)

@zedcodes Ай бұрын

@@bobfreeman7349 That worked, thank you. I used this and it worked: pil_image = pil_image.resize((28, 28), PIL.Image.LANCZOS)