Deploy Hugging Face models on Google Cloud: from the hub to Vertex AI

In this series of three videos, I walk you through the deployment of Hugging Face models on Google Cloud, in three different ways:
- Deployment from the hub model page to Inference endpoints ( • Deploy Hugging Face mo... ), with the Google Gemma 7B model,
- Deployment from the hub model page to Vertex AI (this video), with the Microsoft Phi-2 2.7B model,
- Deployment directly from within Vertex AI ( • Deploy Hugging Face mo... ), with the TinyLlama 1.1B model.
Get started at huggingface.co :)
⭐️⭐️⭐️ Don't forget to subscribe to be notified of future videos. Follow me on Medium at / julsimon or Substack at julsimon.substack.com. ⭐️⭐️⭐️

[@spicule123]

This is fantastic!

[@juliensimonfr]

Yes, I like it too :)

[@SO-vq7qd]

Thank you!

[@juliensimonfr]

You're welcome!

[@recaia]

If I upload a code like this and I want to send and receive data through my website on my website, which is programmed in Python, and on another host, how is that done? Or should I create an API and upload it to Cloud?

[@juliensimonfr]

The endpoint is just another HTTPS API. You can invoke it from your app. This blog post has a good example medium.com/@ashika.umanga/deploying-custom-fine-tuned-llms-on-vertex-ai-6f96752f9fc1, see the "Connecting to the endpoint and performing inference" section.

[@barber5937]

Can I deploy my own huggingface model to vertex? My model says it is endpoints compatible but I don't see anything indicating vertex ai compatibility or how to achieve that

[@juliensimonfr]

Yes, assuming your model is based on a supported architecture (Llama, etc.). If you create a model card with the appropriate tags (architecture, task type, etc), the "Deploy" button will let you deploy to AWS, Google, etc. See huggingface.co/docs/hub/model-cards for more, as well as model card for well-known models (google, meta, etc.)

[@barber5937]

@@juliensimonfr Awesome, thanks julien