Deep Dive: Hugging Face models on AWS AI Accelerators

Explore the technical intricacies of optimizing Hugging Face models on AWS accelerators in this detailed walkthrough, possibly the most complete and most up-to-date available today.
⭐️⭐️⭐️ Don't forget to subscribe to be notified of future videos. Follow me on Medium at / julsimon or Substack at julsimon.substack.com. ⭐️⭐️⭐️
This video focuses on the hardware and software details essential for achieving peak performance. Access relevant code snippets and developer resources, suitable for both newcomers and experienced professionals. Whether you're familiar with Trainium and Inferentia2 or approaching these technologies for the first time, this technical walkthrough ensures your readiness for success in deploying Hugging Face models on AWS.
Dive into all key components!
05:00 AWS NeuronCore-v2
10:30 AWS Trainium
13:48 AWS Inferentia2
16:25 Amazon EC2 Trn1
20:12 Amazon EC2 Inf2
23:20 AWS Neuron SDK
30:00 AWS Neuronx Distributed
35:25 AWS Transformers Neuronx
41:41 Hugging Face Optimum Neuron training and inference
Links:
- AWS Neuron SDK: awsdocs-neuron.readthedocs-ho...
- Hugging Face Optimum Neuron: huggingface.co/docs/optimum-n...

[@TheDarco78]

Bravo Julien! Very nice!

[@juliensimonfr]

Thank you :)

[@viejoven8877]

Thanks a bunch. For a "lean and mean" programmer like myself, having dabbled with the hugginface libraries (high level), it's really good to understand the underlying stack. I'm trying to understand as deep as I can, this is a really useful resource.

[@juliensimonfr]

Glad it helped!

[@LindsayHiebert]

Great job!

[@juliensimonfr]

Thank you! Cheers!

[@bryanguillerm5864]

Merci !!! 🙂

[@juliensimonfr]

Avec plaisir !

[@jobautomation]

Thank you!!!!

[@juliensimonfr]

You're welcome!

[@ChrisSMurphy1]

Julien back to bringing the goods...Applied for a BDR position at HF, waiting to hear back.