Transformer training shootout, part 2: AWS Trainium vs. NVIDIA V100

[@nathanbanks2354]

It's nice to see someone running larger instances. Most videos are about running on your home PC with a GTX 3090 or using an external website from Open AI or Eleven Labs or whatever.

[@idansimantov]

Great vid! I hope you will share an updated benchmark with AWS Trainium 2 vs Nvidia H100 that will be checked by GPT4 instead of GPT2.

[@chrisdwalton]

Nice - can you compare Trainium with P4d (NVIDIA A100) next?

[@mvasa2582]

@juliensimonfr thank you for working on this. I just ran across your benchmarking. It was very well thought through and excellent analysis. Just curious, like the other request below, have you compared Tranium to A100 A100-80 or H100. I understand that there may be huge costs associated with this - Could you please tell us how much this typical run costs? One more quick question - you captured 5.20 Hrs on V100x8 vs. 2.21 Hrs on Traniumx16 - what were the final numbers when this was completed. Was it exactly that? Please advise.

[@gxhoi]

Next time, can you show tutorial for llm + triton + inf2. for llama 2 ?