Azure ML: deploy Hugging Face models in minutes!

In this video, I show you how to deploy Hugging Face models in one click on Azure, thanks to the model catalog in Azure ML Studio. Then, I run a small Python example to predict with the model.
⭐️⭐️⭐️ Don't forget to subscribe to be notified of future videos ⭐️⭐️⭐️
To get started, you simply need to navigate to the Azure ML Studio website and open the model catalog. Then, you can click on a model to select it. This will initiate the setup process, which takes care of all the required infrastructure for you. Once the setup is complete, Azure ML Studio provides a sample program and you can start testing the model immediately!
If you want to experiment with the latest state-of-the-art models, Azure ML Studio provides a hassle-free way to do so. Try it out and explore the possibilities of cutting-edge AI models with just one click!
Azure ML : azure.microsof...
Follow me on Medium at / julsimon or Substack at julsimon.subst....

[@daniellasek1123]

Hey, did the whole interface change? I don't see the same things you do in "Azure AI Studio". It only has nav items too: Home / Explore / Build / Manage. If you go to "Explore -> Hugging Face" you can only select from a predefined list of models.

[@juliensimonfr]

No idea...

[@vincekst]

nice t-shirt Julien!

[@joejay6389]

Hey, can you also show how to deploy a custom model not on the model cards?

[@juliensimonfr]

If you're looking for a one-click solution, please check out huggingface.co/inference-endpoints. If not, the Azure ML doc has some examples: learn.microsoft.com/en-us/azure/machine-learning/how-to-deploy-models-from-huggingface

[@rafaeel731]

That's why I clicked on this vid

[@buksa7257]

H, nice vid. I got it working. Maybe a bit specific question: Now i'm trying to use langchain with it, but all langchain chatmodel classes (openAI, mistral, olllama etc) use hardcoded endpoint ".../v1/chat/completions" while Azure generates endpoint ".../score". Also when using a local model via LM Studio the prebuilt server also already points to this /v1/chat/completions endpoint. Should i make a custom class extending baseLLM to point it to my custom azure endpoint? or is there already something built for this i can use out of the box?

[@juliensimonfr]

Hi, thanks! Not an AzureML specialist at all, but have you looked at python.langchain.com/docs/integrations/chat/azureml_chat_endpoint ? It looks like you can pass the ".../score" endpoint.

[@buksa7257]

@@juliensimonfr ah, thanks! I've been looking though the JS documentation but apparently its only supported for python atm

[@markthompson1439]

Hi Julien, I'm a product analyst working with my manager to propose a new AI based product for internal use. We are not a tech company. Can you advice how to know how big an instance would I need to run LLMs with 7B to 14B paramterers. Just ballpark ideas will do for me. My intention is to give a ballpark cost estimate before we commission a feasibility study with a software vendor who'd give us more accurate numbers

[@juliensimonfr]

Hi Mark, for small scale projects, you can run 7B models on T4 GPUs, which are available on Azure in NCasT4_v3 instances. Realistically, for most 7B/13B models with some scale, you'll need A100 GPUs, which are found in NC A100 v4 instances. How many instances you'll need depends on how much traffic you want to serve, and a deeper evaluation is required. You'll find pricing on the Azure website.

[@caiyu538]

👍