You don't need Supercomputers for AI!

Do you really need ridiculously expensive supercomputers to run your enterprise grade deep learning workloads in production or could you be just fine...or even better off with just traditional servers?
Well, the answer is not that simple. Let me explain!
💥 NEW ONLINE COURSE: "Introduction to Enterprise IT [2024]" 💥
To learn the fundamentals of the entire Enterprise IT, enroll to my new 9-hour Online Course "Introduction to Enterprise IT":
academy.techenthusiast.com/p/...
======================
📜 Table of Contents
======================
00:24 Setting the stage
01:05 Understanding enterprise AI projects
02:10 Servers or supercomputers?
03:43 The answer is: you need both
04:40 Public cloud gives access to HPC
05:51 HPE offers best of both AI worlds
07:28 Executive Summary
======================
🌐 Relevant resources
======================
Frontier supercomputer:
www.olcf.ornl.gov/frontier/
LUMI supercomputer:
www.lumi-supercomputer.eu
HPE GreenLake for Large Language Models:
www.hpe.com/us/en/hpe-greenla...
======================
🎵 Music
======================
Epidemic Sound
www.epidemicsound.com/referra...
======================
💼 Business Enquiries
======================
Would you like me to review/introduce your cool data center solutions?
Contact me: markus[at]techenthusiast.com
======================
🔗 Follow me
======================
Tech Enthusiast Academy: academy.techenthusiast.com
LinkedIn: / techenthusiast
Website: techenthusiast.com

[@TechEnthusiastInc]

💥 Check out my NEW COURSE "Introduction to Enterprise IT [2024]" and learn the fundamentals of Enterprise IT in one go and one day! 💥 academy.techenthusiast.com/p/introduction-to-enterprise-it

[@aryehbarron4067]

Great video as usual Sir!

[@TechEnthusiastInc]

Thank you very much, Ary! 👍

[@svrangarao1224]

Thanks for the information

[@TechEnthusiastInc]

My pleasure!

[@RossCooper-Smith]

Another critical need for AI training is that the source data set has to be on flash. It's a small I/O, random read workload, across the entire training dataset. If you do the math, a single NVidia H100 GPU needs the IOPS of around 8,000 hard drives to keep it busy. You can't even use hybrid storage since caching doesn't work for totally random I/O. NVidia have a certification program for storage arrays, and for AI Training they don't certify anything that isn't all flash. And I totally agree with your conclusion, HPE have a really strong portfolio for AI workloads. They're the only vendor with a full stack in-house solution proven at every level of AI training and deployment. From Supercomputing to Cloud, Datacenter to Edge. They've made some very good strategic choices with their focus over the last few years.

[@TechEnthusiastInc]

Hi Ross! You are absolutely right, very good point! And thanks for all the detailed specs, appreciated. Indeed, there are quite a lot of critical requirements with AI compared to traditional workloads. Agreed. HPE has all it takes to be one of the key players in the field and it has shown it’s bold enough to make the needed risky first moves too. Very interesting to follow! By the way, congrats on the super interesting VAST Data Platform announcement! We need to get back to that. 😉

[@papi-jor9239]

Hi Markus, Thanks a lot for your informative video again. Is the data stored with HPE Greenlake for AI in your own datacenter or in a datacenter of HPE? Thanks!

[@TechEnthusiastInc]

Thanks, my pleasure! The data will need to be stored close to the CPUs and GPUs for the fastest access so it needs to be stored within “HPE Cloud”. In North America HPE is using a Canadian co-lo service provider called QScale where they are building their HPC/supercomputing fascilities and all the training data will be located there. I actually just made a video about all this. Check it out! HPE just announced AI public cloud! (with HPE GreenLake for Large Language Models) kzbin.info/www/bejne/mn6aeauZhbJ3kNE

[@RossCooper-Smith]

HPE GreenLake for Large Language Models is an HPE AI Cloud service that runs within HPE's own datacentres, but they also have on-premise solutions all the way from Enterprise to Supercompute scale. For example: HPE GreenLake for File (GL4F) is an on-premise enterprise solution running a software stack that's already proven for Top-10 HPC workloads and some of the worlds largest AI Clouds. There's a UK deployment of that stack running a 60PB single namespace for data and 100,000 Kubernetes containers, all powered by around 15,000 HPE CPU & GPU compute nodes. GL4F will handle on-prem workloads from 200TB to 200PB and beyond. And of course HPE have their Cray supercompute division as well. Doesn't matter if you want on-prem or cloud, large or small, HPE have you covered. :-)

[@ingridstrombeck8221]

TACK!

[@TechEnthusiastInc]

Varsågod! 😆