Lightning Talk: PyTorch 2.0 on the ROCm Platform - Douglas Lehr, AMD

Lightning Talk: PyTorch 2.0 on the ROCm Platform - Douglas Lehr, AMD
Talk about the current state of PyTorch on the ROCm platform. Including efforts to achieve day 0 support for Triton on Pytorch 2.0. As well as performance improvements, efforts with Huggingface, and other areas.

[@wolpumba4099]

*Summary* *0:00** - Introduction* - Douglas Lehr, principal engineer at AMD, focuses on AI model optimization and performance in PyTorch with ROCm. *0:27** - Overview of ROCm and PyTorch* - AMD has a long-standing partnership with the PyTorch Foundation. - Focused on optimizing PyTorch on AMD GPUs. *1:44** - What is ROCm PyTorch?* - Essentially PyTorch but adapted for AMD GPUs via the ROCm platform. - Can be installed from the PyTorch website, Docker Hub, or built from source. *3:26** - Enhancements in PyTorch 2.0 and Beyond* - Added HIP graph support for better kernel launches and API calls. - Introduced day-zero support for Triton on ROCm with PyTorch 2.0. - Added HIP solver support for linear algebra operations. *4:49** - Triton Enablement* - Added Matrix Fuse Multiply instructions for better performance. - Achieved significant speed-ups in Hugging Face models with no code changes required from the user. *7:05** - Supported AMD GPUs* - Compatibility extends to a range of AMD Instinct and Radeon GPUs. - Announced support for new Radeon 700 XTX and Radeon Pro W700 GPUs with ROCm 5.7. - 8:06: PyTorch nightly supports 700 line of GPUs. *8:03** - Availability and Setup* - 8:09: One PyTorch version supports both CDNA and RDNA GPUs. [edited: Thanks ickorling7328] - 8:18: No need to download different versions for different GPUs. - 8:36: GPUs are new releases, info added last minute. - 8:46: Visit Rockm radiant start page to set up AMD GPU kernel. - 8:55: Install Rock SDK as per instructions. - 9:02: PyTorch setup steps can be found on Rockham docs or p.org for nightly wheels. *9:24** - Benefits and Features* - 9:39: Code compatible across different GPUs with Rockham. - 9:46: 700 XTX has 24GB and 700 has 40GB of RAM. - 9:57: Supports various models like Fusion Llama V2 out-of-the-box. *10:16** - Practical Implementation* - 10:43: Longest part of setup is downloading stable diffusion. - 10:51: Personal experience generating images on 700 XTX at home. - 11:13: Easy to set up with AMD GPU kernel driver. *11:17** - Conclusion* - Ends with thanks and invitation to ask questions at AMD Booth.

[@ickorling7328]

Important revision: 8:09 speaker actually says CDNA and RDNA GPU's RDNA3 architecture is new, ots on both dedicated GPU's and integrated APU's. I hope they mean that it works on RDNA3 APU (integrated graphics) because that mean Mini pc's with RDNA3 are good for running AI things, albeit kinda slow, but power and cost effective. 🎉

[@47Lancelot]

Please, don't forget about windows 10/11

[@billyblackburn864]

Why does only 7900xtx/x get official support

[@galasitygaming]

Guess we have to get Nvidia GPUs

[@yorozuya61377next]

I want it to work for windows

[@changminoh1]

예제를 하려면 8k이미지 이상으로 테스트 하는 화면을 보여줘야지 그래야 확인하지