Intel's Big AI Bet: Gaudi 3 ft. Supermicro -- System Tour, Tested Pytorch & More

[@eltamarindo]

The factory where tinnitus is made!

[@glenn3378]

elevenitus!

[@dennisfahey2379]

The funny thing is its not the airflow of the fan - its the vibration of the rotation coupling to the big chassis that acts as an acoustic waveguide. I did hear of a company who went as far as to suspend the fans in a rubber and modify parallel fans speeds to not be additive. They even did a speed table and acoustic measurement to determine what tones different fan RPMs produced and kept away from harmonics that would be perceived as harsh to the human ear.

[@kellymoses8566]

@@dennisfahey2379 Seems like you could tune the fans RPMs to have destructive interference in their sound output.

[@re5onance244]

@@kellymoses8566 you could, but will it impact pressure is the big question mark there

[@SimonZerafa]

Tinnitus? More like hearing loss for anything more than a short period of exposure. Extremely poor cooling design if no thought was paid to this.

[@DaxHamel]

I'm only halfway through and I'm really enjoying this interview where you have enough time to have an actual conversation.

[@kellymoses8566]

Listening to two smart people talking about something they know a lot about is always fun.

[@Examoon]

I'm hoping to see an intel arc card targeting home user AI and gaming with at least 32 GB of vram in the $350-700 range. If intel can make a high vram consumer card thats affordable they can probably pull some sales that would have otherwise gone to nvidia's 5090.

[@ToroidalVortices]

32GB of normal RAM is 300$. You wanna slap a tensors cores on top of that for 50$?

[@lbgstzockt8493]

@@ToroidalVortices You are off by a factor of two, unless you want the best HBM there is.

[@ToroidalVortices]

@@lbgstzockt8493 GPU's use GDDR6/7 - so yeah, I'm basing it off highspec DDR5 because its the most price comparable raw chip to raw chip.

[@dizzident]

I'd pay 1500 for an Intel reference card with 64gb+ of memory

[@arenzricodexd4409]

Pipe dream. Intel will not going to pull such move for some gamer and niche AI hobbyst. Want more VRAM? buy true pro card.

[@Maxjoker98]

I think the fact that they actually do the inter-node IPC using Ethernet is really clever, and not just for existing infrastructure! I'd bet that the faster Ethernet standards are going to survive long-term and will eventually be standard for every kind of compute. Investing in (expensive, high-end) ethernet infrastructure is a long-term general-use investment, while investing in nVidias's proprietary inter-GPU connect is only useful for training with a specific generation of nVidia GPUs used for ML training. I assume any new datacenter chooses Ethernet basically by default, and only switches(ha!) away from that when they need to(which was the case with nVidia, but apparently doesn't need to!), which is a great advantage for Intel IMHO.

[@prashanthb6521]

Do you think the latencies are acceptable in the ethernet compared to nvidia's proprietary inter-GPU connect ?

[@jonogrimmer6013]

SuperMircro 'That's not a fan, THIS is a fan!'

[@initdeit]

I enjoyed your comment on hearing protection through the continuous loud "EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE" in my left ear.

[@MrHav1k]

This is very, VERY cool Wendell! Love how you actually mentioned how simply it is to get the Pytorch code running, mentioned Supermicro's Jumpstart, etc. Very nice!

[@kchiem]

1:55 I want to know more about that fan at the wall setup.

[@someusername121]

That style of fan has a larger intake than the exit, so it’s blowing between the racks.

[@spx2327]

That is a portal to Pandora

[@darklordofapathy]

That's a portacool evaporative cooling fan.

[@edmicheals]

upgraded 14900 heatsink fan

[@dalehorton7748]

@@edmicheals it still overheats

[@russha5891]

Wow, as an ad for Supermicro, I did not mind at all. every point was honest and backed up with info to support it. I liked the yes I had troubles but could solve them, not everything is perfect but with effort it can be solved. Thanks Wendell

[@xyzabc123-o1l]

1:50 IM HITTING MY DMT CART RIGHT NOW BRO IM BREAKING THROUGH

[@Hawlkeye-e9p]

Lmfao Hero lvl launching

[@Jango1989]

Spot the commentators that have never been in or even watched videos of a data center lol. 😂 There's a reason why ear defenders are handed out like candy at the entrance in most data centers.

[@breakupgoogle]

we have a db meter in our cage. the min noise 20' from a rack is 81db. you have to have hearing protection

[@Hawlkeye-e9p]

Thats not wrong. Cold af, loud af, super annoying. I spend the least time possible in there. Get in get out. Worst part of all is being in there when your hungry. Its pure torture

[@breakupgoogle]

@@Hawlkeye-e9p so dry also.

[@lbgstzockt8493]

@@Hawlkeye-e9p Sounds like a headache generator

[@nemesis851_]

Glad my DC isn’t anywhere near that loud. Sure there are loud areas, but generally it’s a fine place to be. Of course noise cancelling earbuds make it better!

[@breakupgoogle]

im a supermicro fan. they are the Chevy LS of servers.

[@timramich]

They will eventually get to the point of being all proprietary and parts won't fit across generations.

[@ABaumstumpf]

Oh yes, your normal everyday 800Gbps Ethernet. Let me check again... no, no even 100% of the memory bandwidth in our build-server can feed that.

[@Level1Techs]

yes but rdma is magic. so hbm2hbm your memory doesn't need that much bandwidth:)

[@runforestrunfpv4354]

you should see the size of the 800GB optics lol.

[@divyjotsingh3879]

This was awesome. Felt more like a casual conversation discussing the product, and I wouldn't say no to that. I just hope Intel continues down this path with falcon shores

[@kchiem]

20:30 what's up there Wendell?

[@abavariannormiepleb9470]

Are those the systems gobbling up all the DDR5 memory chips with the density that would be needed to build 64 GB ECC UDIMMs?

[@dirkmanderin]

Probably, just like AI systems have caused HDD prices to go through the roof.

[@ThatGuy-ht9sp]

Thanks Autumn, back to you Wendell

[@blahorgaslisk7763]

The sound of the system was pretty typical of what you expect for at least start up noise for a large server. The usually slow down once the function of every fan has been tested and temperatures has been found to be within spec. But the sound of full RPM fans are always nasty. I worked with building servers for way too many years and suffer tinnitus as a result. When working on these every day you tend to get lax with the ear protection. Don't do that. Always use ear plugs or better protection. That whistling in the ears totally suck and at time it makes it har to sleep as they start beeping and whistling in your head. Same with using ear buds for listening to music all day. Be aware that the so common earbuds inject sound directly into your ear and tend to produce way more sound pressure than you think they do. Keep the volume down or you will damage your hearing. Yes it sucks, but tinnitus and hearing loss sucks even more though you might not experience it before you turn 30 or 40 years old. But when you get the problems it's too late to make anything about it. The damage is already done and you will curse your addiction to loud music...

[@yensteel]

Helped optimise a medium sized cryptocurrency farm before. There, the noise is continuous all day and night with the asics, and the room temp is over 40c. Glad the consulting was only for a few weeks. Hearing protection is essential and nothing to snuff about. OHSA regulations have a maximum noise level for different exposures to sound. They're supposed to be followed but a lot of industrial places still don't adhere to them nor do they provide equipment.

[@peq42_]

Hey I heard if you listen to quiet sounds/music on a low volume, over time, it goes away

[@Yandarval]

Double thumbs up for the ear pro advice. Ideally, you want the foam plugs in, as well as noice cancelling headphones. Your hearing is your responsibility. No amount of money can replace it. Its generally the IPMI/iLO/iDRAC etc, not being booted up that causes most of the "fan panic" on a cold boot. if the IPMC has booted up, the fan test can be just a few seconds on a cold boot. As the IPMI is there to control the fans. Easier said than done of course, in production. If you can plug in power and management to a server and leave it alone for 5-10mins. The IPMI is all booted up in the off server. The fans "should" be more civilised once turned on.

[@blahorgaslisk7763]

@@peq42_ You wish that was how it works, but a part of it is right. If you listen to real loud musing and discover that after wards you hear a beep in the ear then letting it rest with very moderate soundleves for a time will make that beep go away. At least the first time. This is one reason people tend to think that they are invulnerable when they are young. That beep went away so it wasn't a problem. I'll just do the same if it ever happens again, no problem... Sure it works for a time, but as the years go by the body will get worse at repairing problems like this. Sure the beep might go away but you lost hearing of certain frequencies. Didn't notice that? How about the mosquitos? I remember when I was younger I could hear them all the time. A mosquito in the bedroom? So damn annoying hearing it fly around the rom. Hear the crickets? I realized that it was many years since I last heard a cricket, and they were everywhere during the summer. You can lose a lot of frequencies and not really notice as the brain adapts and make it easy to still hear the important parts, such as voices and talking. That is until you damage your hearing real bad by not protecting yourself as "it's no big deal it goes by" and then suddenly you start to realize that something is wrong. People reacts to sounds that you never hear and so on. Eventually people start mumbling and are harder and harder to understand when talking to them, and then it's to late to save you hearing and you have to live with it. Now let me say that living with hearing problems suck. I got the extra special version a year ago. Falling down a stair I crushed my head and woke up in the hospital two weeks later deaf on one ear and my honestly earned damaged hearing on the other. It sucks even more now. So be careful about your hearing. Once it's damaged it doesn't come back. And the damage might not be all that serious in the beginning but it adds up. It doesn't really heal, remember that. Some problems can get a bit better, but it doesn't heal 100%, it only add to previous damage and gets worse. Add that with age hearing flaggs because of age. But you don't have to hurry it up, you certainly don't want to hurry the hearing loss. Once you start to notice the problems it's too late, and you will be pretty pissed at your self for not taking care of your hearing back when you could have done it.

[@EyesOfByes]

This stuff is insane when you consider that the entire videolibrary of KZbin in 2006 was 40 TeraByte. In total. The entire thing. Source: Gueniess World Records 2008

[@perhansen3959]

Nice little fan in the background is it a phanteks ? :)

[@dirkmanderin]

I always loved new toys, but its nice not having to worry about power and cooling working with cloud systems.

[@yveice]

Can't wait, until airbus use Supermicro servers instead of Rolls-Royce turbines to make planes lift up. 🤣

[@kornydad14]

I'd really like to know what the performance is vs AMD and Nvidia systems currently available.

[@K4g4m1]

We're gonna need a bigger fan.

[@ecash00]

it would be interesting if you could give us a location that collects the OLD hardware, that MAYBE the consumer can get hold of. Those OLD networks Still work and are Still better then MOSt things the consumer can get, and have a Better Life expectancy.

[@SimonZerafa]

Almost certainly that would be too loud for use in the UK also. Very good ear protection at a minimum. High frequency hearing loss is NOT fun. Ask me how I know 🤨🤷‍♂️

[@thereddog223]

I would be interested to see how it works

[@RyanMelena]

Are there Gaudi 2 PCIe solutions available for workstation / homelab use? Would definitely be interested in something like that if it too was on "fire sale".

[@runforestrunfpv4354]

Super Micro is going on all outmedia blitz lately.

[@zm1he-zm1he]

I can't wait til MMO dev's get a hold of this tech. Imagine GPU server side physics checking, tons of cores for connections, sharding worlds in the same rack with DMA/CXL. Network stack sounds strong enough to push large amounts of data to sync to the client. Or do some gaming MMO VDI and have it all in the same DC and only send down gameplay frames so sim can send massive packet sizes within the DC net for game state. One can dream...

[@GeekProdigyGuy]

It'll take another 10 years.

[@noprivacyverner]

jensen was in denmark to open a new AI Super computer the other day

[@RedSntDK]

Which Jensen? Jensen is the most popular last name here, so which one?!

[@BestjeJust]

6:20 It's always DNS

[@Whalerguy]

"an internet rando" Lol A bit more prestigious IMHO.

[@Emerson1]

Great video!

[@ZippydsmLee-freetube]

I guess if you push the silicone hard, you can make huge leaps every year or so.... data centers have better cooling, so its harder to notice.... hehe....

[@RedSntDK]

It's no joke, put on protective gear ffs. Tinnitus isn't fun.

[@michaelrichardson8467]

Impressive feat for SuperMicro and Intel.

[@MauraWhiffen]

Yoooo this fire

[@Veptis]

It would be dope if they had a pricetag for the PCIe card. I need a solution to run 34B inference for my workstation... At BF16 or better, no quantization. Intel got a big win with PyTorch 2.5 now supporting Intel GPU natively, even on Windows (ROCm still doesn't). And this in ludes iGPUs, Arc and PVC... But there was zero marketing celebration for that. It should be much bigger news. Intel marketing has only shrunk to 70%, someone are around. And they aren't reaching developers either. maybe new dGOU hardware first(really soon). Also not Gaudi. Habana still has their special libraries and exporters, not part of accelerate just yet (which matters for general inference via transformers and a ton of dependencies). It might be possible via dynamo.... I saw the RFC already.

[@yensteel]

Gaudi 3 is around half of the cost of the H100 80gb card according to news. So, it's around 10-20k as the H100 is 30k-40k on ebay. It's not an accurate figure for large tech companies, as they ordered in the thousands each.

[@Veptis]

@@yensteel well, Intel has said the 8x Gaudi3 OAM box costs 125k but a single card + water block should be cheaper. It's not marketable towards workstation due to the idle power requirement - but being able to buy it in the first place would be a good first step.

[@theworddoner]

I want to get my hands on a gaudi work station gpu. I’d love to buy 64gb+ vram gpus to use in my workstation.

[@ffsireallydontcare]

Being air cooled at least it might have a secondary market use and stay out of landfill longer than the nVidia water cooled manufactured ewaste. Talk about pump and dump.

[@mrd5024]

Me watching this on my Core 2 Duo

[@MikeBob2023]

Wow. My head is spinning. *Do not* let Pinky and the Brain get anywhere NEAR that thing. 😳 EDIT: P/S: Thank you, Wendellman! 🙏🏼👍🏼

[@Hawlkeye-e9p]

Final nail in the coffin. Everyone is starting to hate ai anything as they try to shove it down our throats. The big gamble is going epicly fail

[@TazzSmk]

makes me wonder if AI boom will also result in AI-Internet boom, I mean, it's likely to see some kind of P2P open-source compute networking to effectively aggregate all the tech, those 300K servers in the video are "not enough" but also not "affordable enough" to scale up by single entity - and from customer perspective also not to rely on single cloud compute provider like AWS (even if they build new datacenters)

[@karehaqt]

Maybe Supermicro can use Gaudi powered AI to figure out a way to appease the DoJ

[@dennisfahey2379]

Famous last words - "It's all air cooled". Ask any lab manager what happens when you overload the building cooling - continuously. No wonder Google was shopping around for power and natural cooling options when picking new data center sites decades ago.

[@lbgstzockt8493]

I wonder what this hardware could be used for it the AI boom ever turns into a bust. Massive amounts of matrix performance surely have other applications?

[@DoNotFitInACivic]

Curious if these style "gpu" s could be used for rebder farms, or re tasked as HPC jr.

[@EdDale44135]

NVidia is sold out on their AI product. AMD is close to sold out. Leaves room for this assuming it can be used to meet the same need.

[@maxstrong1999]

But does it work out of the box with pytorch/Tensorflow?

[@Level1Techs]

@maxstrong1999 yes if you get the gaudi Habana version which is close to being ready for upstream

[@Level1Techs]

gaudi 3 has the added bonus if actually being legit, too

[@maxstrong1999]

@@Level1Techs good to know. Definitely a huge issue I've run into is the non Nvidia tooling being subpar.

[@EdDale44135]

@@maxstrong1999 Wendel tested it with pytorch.

[@FLOODOFSINS]

They are not flying you out there for a bad review😂

[@Kurukx]

2016 glue was a long tine ago... Good luck intel !

[@robmafia3915]

if only they let you see their books

[@dinckelman]

The only AI-accelerated thing i’m looking towards is the disappearance of this pump and dump scheme, just like what happened with crypto and nfts

[@chebrubin]

I told you this would happen. Intel hitching its AI wagon on a company under investigation. Didn't AAPL stop buying Supermicro motherboards 7 - 8 years ago according to the WSJ?

[@karehaqt]

Super Micro share price is cratering, down 30% because Ernst & Young has resigned as their auditor due to "information that has recently come to our attention which has led us to no longer be able to rely on management's and the Audit Committee's representations and to be unwilling to be associated with the financial statements prepared by management". The silence from the tech press on this ongoing situation is deafening.

[@InvadersDie]

Are you telling me that this system in about 4-6 years time will be less powerful than my phone?

[@lemmonsinmyeyes]

I like this dude

[@vonWeizhacker69]

Ja mei is des a gaudi!

[@stevetheborg]

Wish intel would build a GPU from this thing. we need it to edit samsung cell phone video /s

[@shazzz_land]

I think it is time for apple to make its 'entry' into HPC compute gpus, id have a feel they d kill the market

[@borgheses]

so, it sounds like my pc...

[@ConorHanley]

How many Tulips equals a Guaudi 3?

[@ullibowyer]

18:40 This discussion about storage doesn't make a lot of sense in the context of Gaudi 3. Gaudi isn't good at training, and you only really need storage for training. For inference you don't really need either training data or model checkpoints so a lot less need for storage.

[@Delease]

I found this video to be a bit sales-y. That being said, I am genuinely excited about Gaudi. The Nvidia boot heel is getting very uncomfortable on my neck.

[@peterl545]

What's the Dell product? Super Micro could get nuked by the SEC at any moment.

[@NyneIX9]

Gaudy, indeed!

[@nemesis851_]

800 Gig per channel, while millions don’t even have 1 Gig to their homes

[@Wanavarbel]

Soyjack thumbnail 😭

[@Brommes]

It's impressive .. But really .. who needs this ?

[@Rushil69420]

Had no idea Tom Segura worked at Intel

[@Nobody-vr5nl]

Meanwhile, intel share holders have been crying that intel hasnt focused on ai enough.

[@SavageBits]

The shareholders are correct: H200 crushes Gaudi 3. While Intel starts volume Gaudi 3 vol production, Nvidia start Blackwell vol production, which is 2 generations beyond Gaudi 3. Topping it off, most of Intel's Habana Labs talent quit when they heard Gaudi 3 was a dead end product.

[@shinjipyon]

I hope its not too gaudy

[@slimjimjimslim5923]

Can they call it something more cool sounding than gaudi. 😂

[@jojosworlds1208]

Funnily enough Gaudi, means fun in german dialect.

[@blackburd]

Put a killawatt on it though.

[@shazzz_land]

Can someone please enlighten me about rdma channel meaning?

[@radeksparowski7174]

but can it run crysis?

[@labor4]

but can it run SuperMario?

[@user-vsmsdos]

Did you soyjack a gaudi3 system? 😂

[@dennisfahey2379]

Tensorcores are not complex and Intel has very good engineers (but bad management/marketing IMHO). So a fast chip roll schedule is not out of the realm of possibilities. Several studies have indicated that feeding these beasts is a bottleneck so it may NOT be Gaudi itself that is faster - it may be WHAT'S AROUND Gaudi that alleviates it from waiting to get busy. I've been watching Gaudi (formerly Habana Labs) since the acquisition and Intel really broke its cadence. (Intel does this on almost all acquisitions. Very bureaucratic.) If anyone can take on NVIDIA, who is benefiting from pole position, it would be Intel and AMD who have equal stature in the DataCenter. Easy for them to fight eachother vs an ankle biter with superior tech who has to get a foot in the door.

[@ethr95awd]

go go decel europe dont change

[@sativagirl1885]

Q: please test/compare latency of 400Gb vs 1Gb ethernet

[@PerpetuallyTiredMusician]

Want to get cool tinnitus like your favourite musicians with out having to rehearse for a few years next to a hi-hat? We got you covered.

[@ZerqTM]

when i first saw the title and Gaudi 3 i though sequal to Gandi II ? kzbin.info/www/bejne/apbKkmiImMinaNU My brain drags out UHF references a bit to easily lol

[@Iam_Dunn]

But, you know, the real question is “Will it run Crysis?” …. 😂 ❤ :)

[@chebrubin]

LOL Supermicro full interface of all LLM inputs back to the PRC intelligence mainframes.

[@intothebeyond8763]

Gaudi has to be the worst name for a computer system in history.

[@tbomblamo]

wendel talking to pretty computer girl

[@DavidWB147]

Intel were getting slaughtered for high temperature and power usage. So they have reduced those, and replaced it with... decibels?

[@Gersberms]

One of the numbers has to suck, or else it isn't Intel... they put all the suck in the sound it seems.

[@ypey1]

It is now turbine cooled

[@rgbplague7834]

That's pretty much how any data center sounds...

[@drewlarson65]

Another l1 video without links.. nice... *grumbles and googles supermicros channel*

[@xephael3485]

Wow more paid promotion

[@JRandomRU]

I'm sorry, but this "oh, it's 13kW per chassis but it's so efficient it'll run on less than that" is total bullshit. From a system integrator/architect perspective, when you see something like "8x3kW PSU (2N redundancy)" - you have to design with that in mind, not whatever marketing numbers are. Which means at least 25kW per rack per chassis. No shortcuts.

[@superslammer]

huh....

[@claucmgpcstuf5103]

yea shure eatith .. no real use expet AI ..IT IS NOT WAT IM INTERTED IN . NOPE

[@guy_autordie]

Wendell?!?! "yeah america, I guess" about the lack of workers' rights? I hope the point of sarcasm I felt in it was real.

[@borgheses]

can it play fortnite?

[@amigatommy7]

"serve the home" channel just showed a tour of Musk's data center.

[@miyagiryota9238]

and?

[@amigatommy7]

@@miyagiryota9238 awesome engineering and power.

[@MelroyvandenBerg]

I don't want Python or Pytorch. Did you know that Python uses 45 more time and energy than programming language like Rust, C++ or even Golang??

[@Level1Techs]

lol did you know that it's not really relevant as a practical matter? because where your ai pytorch program spends most of its time executing code is... not python

[@johnbilodeau6296]

Just looking at pytorch's github and at least 37.4% is written in c++ without even considering submodules and dependencies to other c++ libs. Obviously such a loss would not make economic sense and pytorch would not be so popular.

[@zCaptainz]

Wendell is practically an Intel marketing bot, at this point...

[@MyAnimeTL]

I mean more competition is never bad -> sure it's nothing groundbreaking but also not that bad -> when everything else is nearly sold out

[@zCaptainz]

@@MyAnimeTL who are you talking to?

[@Valk-Kilmer]

Still living as if Xeons are industry leading. Never forgetting him retweeting the Quarterting who is a literal fucking Nazi

[@kemaldemirel1714]

@@zCaptainz Yeah I loved when he marketed those 14900k's by saying they had %99 defect rate. I immediately put in an order!

[@rgbplague7834]

Nope, Wendell is pretty objective and balanced.

[@Phantom-mk4kp]

Why do you insist on calling everything an 'ecosystem' it's not it's just a plain old system. An ecosystem is an area where animal, vegetable and organic entities along with weather and other influential components form the result.

[@kittysreview9055]

Who would buy this slow garbage? AMD MI325X and Nvidia B100/B200 will eat this for lunch!

[@rgbplague7834]

It is not slow for inference, especially when factoring in its cost.

[@kittysreview9055]

@@rgbplague7834 At this scale, cost doesn't mean much to the companies spending millions on this. they will pay a bit more for a ton more perf and most importantly, compute density. Since I can do more inferencing with less hardware due to the higher perf of Blackwell/Instinct, why would I care to get this? None of my customers are requesting it. This thing is DOA. These early adopters who are willing to spend crazy amounts and who are flush with cash are not looking at marginal discounts at that scale!

[@lharsay]

$10K vs $40K