Accelerating Transformers with Optimum Neuron, AWS Trainium and AWS Inferentia2

In this video, I show you how to accelerate Transformer training and inference with the Hugging Face Optimum Neuron library, a hardware acceleration library dedicated to AWS Trainium and AWS Inferentia 2, two custom AI chips designed by AWS.
First, changing a single line of code, I show you how to train a Vision Transformer model on the food101 datasets (75K training images). On a trn1.32xlarge instance, the model trains in under a minute per epoch.
Then, I show you how to export a DistilBERT model from the hub to Inferentia2. Running a benchmark on a inf2.xlarge instance, we get over 2000 predictions per second and P99 1-millisecond latency!
⭐️⭐️⭐️ Don't forget to subscribe to be notified of future videos ⭐️⭐️⭐️
Amazon EC2 Trn1: aws.amazon.com...
Amazon EC2 Inf2: aws.amazon.com...
Hugging Face Neuron AMI: aws.amazon.com...
Optimum Neuron documentation: huggingface.co...
Optimum Neuron Github: github.com/hug...
Code: gitlab.com/jul...

[@caiyu538]

Great lectures. Great teacher.

[@smartin2minutes]

Hi @Julien Simon, thanks a lot for the video. I have checked your code. Looks like something I can use. One question, that I could not find anywhere, is there a way to use 'pipelines` with a Optimum Neuron model? I have a token classification task and pipeline just makes things easier to maintain. Will be very helpful if you have any examples. Thanks.

[@FalahgsGate]

thanks for best video ❤

[@juliensimonfr]

Most welcome