It's nice to see someone running larger instances. Most videos are about running on your home PC with a GTX 3090 or using an external website from Open AI or Eleven Labs or whatever.
@idansimantov10 ай бұрын
Great vid! I hope you will share an updated benchmark with AWS Trainium 2 vs Nvidia H100 that will be checked by GPT4 instead of GPT2.
@chrisdwalton Жыл бұрын
Nice - can you compare Trainium with P4d (NVIDIA A100) next?
@mvasa2582 Жыл бұрын
@juliensimonfr thank you for working on this. I just ran across your benchmarking. It was very well thought through and excellent analysis. Just curious, like the other request below, have you compared Tranium to A100 A100-80 or H100. I understand that there may be huge costs associated with this - Could you please tell us how much this typical run costs? One more quick question - you captured 5.20 Hrs on V100x8 vs. 2.21 Hrs on Traniumx16 - what were the final numbers when this was completed. Was it exactly that? Please advise.
@gxhoi Жыл бұрын
Next time, can you show tutorial for llm + triton + inf2. for llama 2 ?