Accelerating Transformers with Hugging Face Optimum and Infinity

Speakers:
Lewis Tunstall, Machine Learning Engineer
Philipp Schmid is a Machine Learning Engineer and Tech Lead at Hugging Face, where he leads the collaboration with the Amazon SageMaker team. He is passionate about democratizing and productionizing cutting-edge NLP models and improving the ease of use for Deep Learning.
Philipp Schmid, Technical Lead, Hugging Face
Lewis Tunstall is a machine learning engineer at Hugging Face, where he focuses on developing tools for the NLP community and teaching people to use them effectively. He’s built machine learning applications for startups and enterprises in the domains of NLP, topological data analysis, and time series. Lewis has a PhD in theoretical physics and has held research positions in Australia, the US, and Switzerland.
Abstract:
Since their introduction in 2017, Transformers have become the de facto standard for tackling a wide range of NLP tasks in both academia and industry. However, in many situations accuracy is not enough - your state-of-the-art model is not very useful if it’s too slow or large to meet the business requirements of your application.
In this talk, Lewis Tunstall and Philipp Schmid, Machine Learning Engineers will give an overview of Hugging Face’s efforts to accelerate the predictions of Transformer models. They'll discuss a new open-source library called Optimum, which enables developers to train and run Transformers on targeted hardware. They'll also introduce Infinity, which is a containerised solution that delivers millisecond-scale latencies in production environments.

[@Gerald-iz7mv]

Hi, how to export to onnx using cuda?