3000W AMD Epyc Server Tear-Down, ft. Wendell of Level1Techs

[@GamersNexus]

Find Level1Techs here! kzbin.info Grab a GN Mouse Mat or Coaster Pack on the GN store! 10% of all store revenue goes to the charity and cat shelter Cat Angels until 9/17/22 - store.gamersnexus.net/

[@tobiwonkanogy2975]

will absolutely watch the server upgrade. Hyper dense server .

[@timmetimme1360]

So no Crysis 🤔

[@themice42]

Wendell.....LEGEND

[@sketchtherapy1218]

He has a point, it's a pealing.

[@Cinetyk]

*almost*

[@anumeon]

"We don't do that here" - Steve "It's a-pealin'" - Wendell.. :D Good one Wendell

[@rossharper1983]

I do enjoy how Wendell makes every other Tech KZbinr sound like a complete tech novice.

[@sammiller6631]

Wendell doesn't make Dr Ian Cutress of TechTechPotato sound like a complete tech novice.

[@klfjoat]

@@sammiller6631 Ian is ex-AnandTech. AnandTech makes every non-computer-engineer sound like a complete tech novice.

[@DovahDoVolom]

Most other youtubers are explaining for the masses but Wendell has his audience down. The people who want to know the hardcore details.

[@sammiller6631]

@@DovahDoVolom Wendell is for people who want to know the hardcore details in a _concise_ way instead of Buildzoid's hour of rambling

[@yumri4]

@@sammiller6631 though also not go to the white paper and read for an hour either. It is normally only 80 to 100 pages long with diagrams and descriptions that rely on each other so you need 2 or 3 of them open to understand what you are looking at.

[@eddiehimself]

I had the pleasure of seeing part of a supercomputer array used by the late Stephen Hawking for his physics research being fired up. One thing I noticed about it was how it had so many fans running at full speed when starting up, it was almost as loud as a PS4 Pro!

[@GamersNexus]

Wow! That's really cool that you got to see that!

[@eddiehimself]

@@GamersNexus It was really rather neat. It's actually at a massive collection of vintage computers in North Yorkshire, England. I'm sure you could arrange a visit should you ever find yourselves in the UK c:

[@ImNotHere4750]

💀

[@blahorgaslisk7763]

I built servers for a living some years back. One server is obnoxious, but when you have 40 - 80 1U servers in the test room all running torture test with 40°C in the room every fan is running at max speed. You do need hearing protection even if all you are doing is setting up a new server for test. Being of the "I'm invincible" mind I often didn't, and today I have pretty bad tinnitus. So use hearing protection when working in a server center or even when just testing a single server if it's powerful. At worst you're a bit uncomfortable or look like a dork, but at least you can hear what people are saying. Cuts down a lot on the number of times you yell "What?"

[@theroyalaustralian]

Daum

[@TiagoMorbusSa]

Steve going "we don't do that here" has some SERIOUS daddy energy.

[@Craftimonium]

😵

[@Mattspips]

daddy?

[@xenthia]

daddy? sorry... daddY? sorry.

[@twocups8994]

Sus

[@Amberion]

@@cmas5854 a-peeling

[@BenWolkWeiss]

Steve's face and reaction to Wendell telling him the MSRP of the Instinct card at 16:20 is the best. I cracked up and had to watch it a second time.

[@quantumuninstall]

Came to the comments for this. *tries to act cool* *puts it down*.

[@matasa7463]

@@quantumuninstall Looks around nervously. Considering how these things can mysteriously die from nothing, it's a legit concern.

[@PLee-vu6mp]

pure gold.

[@ChristopherWoods]

I had a similar reaction when I found out how much a broken RTX A6000 I was replacing in a work machine cost 💀

[@sebimon4971]

2600W 80+ Titanium PSU in that formfactor is insane

[@jayaskme2942]

Agreed, i remember being limited to a 300w power supply in my 1st and 2nd shuttle builds. 😪. Dont miss those days at all

[@aeasus]

That's for each node and 2 nodes per unit.

[@LtdJorge]

Imagine the price

[@TheCastorPlays]

You know shit just got real when Steve goes "I'm not normally scared of pulling a GPU, but..."

[@blahorgaslisk7763]

Loved the way he put the card down when Wendell said the they retailed at about $7K...

[@markboz3366]

Absolutely, he understands the monetary value, and that most of his audience wouldn't spend half the cost of 1 gpu on their entire system. Such a contrast to another tech tuber who would just rip it out with the attitude of "if I break it I'll just buy it"

[@mkall]

i ve never seen steve flustered in front of hardware before

[@tasnimulsarwar9189]

@@markboz3366 who are you alluding to?

[@jritechnology]

​@@tasnimulsarwar9189 Who cares?

[@michelvanbriemen3459]

Every time Wendell shows up with hot steamy tech it's like TechMoses coming down with another Ten Commandments. I love it.

[@tyguy3876]

Which Moses transferred from a cube to a tablet 😘

[@himmelsrand7527]

Wendell is the best 😂

[@MartinBalle7]

Love the intro 😁

[@treyquattro]

well, Steve's no slouch either!

[@bibasik7]

Crazy how servers that are so densely packed are still repairable, yet Dell can't make their tower PCs with standard parts.

[@zodwraith5745]

Ironically Dell and HP make quite competent servers. It's their desktop offerings that rank below a wish computer.

[@JanMoren]

The budget for designing this hardware is a bit different from that of a mass-market corporate desktop. Also, as these are practically always sold with a support contract, Dell people are the ones who need to be able to fix them, as quickly and efficiently as possible. Probably worth spending the money and time to get the design right.

[@andyruse4670]

@@zodwraith5745 Don’t know about HPs desktops, but their omen laptops are more repairable than other “gaming” laptops. No soldered components, a repair guide that makes it obvious that they were designed to come apart if the end user needed to fix them. I’m reasonably sure their omen desktops even use off the shelf components, saw one tear down feature a cooler master PSU. They just for some unknown reason use a crap tastic case.

[@ConditionsCloudy]

It's not just Dell, unfortunately. HP, Lenovo, Acer, etc. Basically all of them are guilty of it in some models somewhere in their product stack.

[@andyruse4670]

@@ConditionsCloudy For sure, I was just saying that HP has at least one line where they avoid the custom everything horseshit. Dell throws that garbage on everything, Alienware being especially egregious.

[@purplegill10]

I didn't expect this video to be as interesting as it was, and now it's one of my all-time favorite videos from you guys. EVERYTHING in that is just so well-planned out and I can't imagine how much engineering it took to make something so compact yet have an unbelievable ease of use.

[@nutzeeer]

And in 10 years when hardware is 10x as fast the same performance should cost like 7k

[@twogruden9943]

yes it's very a-peeling

[@falxonPSN]

@@nutzeeer yup. It'll cost the same to buy as one month's power bill. The bad news is the power bill won't depreciate.

[@IR4TE]

Steve & Wendell together in a video is always a treat. :D

[@mrmuds8624]

Wendell is quickly becoming "the IT guy" for KZbinrs 😂

[@kojack57]

No no. He's the computer janitor.

[@JBrinx18]

@@kojack57 Especially with that collection of toilet lids, right?

[@GKZanite]

Been following Wendell since Tek Linux started. If there's anyone to trust, it's him,

[@alr6435]

It's crazy seeing Wendell in front of a camera after growing up watching the teksyndicate

[@kojack57]

@@JBrinx18 A very particular bit of knowledge. Although the orange soda ...saga during Covid was very amusing.

[@BRUXXUS]

Wendell’s excitement is so pure and contagious. Any time he’s with GN, it’s an absolute treat to watch.

[@alixxworkshop846]

Im thrilled and amazed how AMD improved their technology. Thanks so our eupwerwomen dr.Lisa Su and their engineers... Been waiting more than 15 years to see AMD logo on my PC and finally got It...

[@DigBipper188]

Can confirm on the double-stacked fan designs that rackmount gear normally uses. They have two sets of counter-rotating blades because you want the air to actually hit the blades and be propelled by them. the issue is though, when air is moved by a fan it introduces some rotation to it, and when you sling another fan in front of that already spinning air and that fan co-rotates, its effectiveness is reduced significantly, and can actually impede airflow while making the fan even louder than normal. so to get around this, you have counter-rotating blades. A lot of manufacturers will also change the blade profiles of one of the fans to ensure a balance of high airflow and high static pressure can be achieved. this is especially important in 1U rackmounted systems as their fans can't be any larger than 40mm, so airflow comes at a bit of a premium. In larger system fans as well, such as 60 and 80mm double-stacked units, the fan may also have additional stator set in the frame that is designed to remove the vorticity generated by the rotation of the fan blades. these styles of fan are normally used in 2U servers especially, as removing that vorticity from the airflow makes it easier to more evenly and accurately split the airflow throughout the system, reducing the chances of a dead-spot in the airflow.

[@or2kr]

Crucially, those fans aren't really redundant as they require both fans to run to deliver the full performance. Usually axial fans just have a pressure-airflow curve that is basically a straight line, but by adding a second counterrotating fan it is able to deliver a pressure-airflow curve like a centrifugal/blower fan, almost doubling the airflow at the operating point. And you definitely need that if you want to cool 3kW in a 2U server. I'm kinda sad that they didn't inspect the fan further, you could definitely try it by hooking it up to a normal PSU, or at least show the model number and show the actual current figure. Because 60W isn't really much anymore, something like the Sanyo Denki 9CRB0812P8G001 has a rated power draw of 110W, four of those is more than what my entire PC consumes

[@DigBipper188]

@@or2kr Yeah, agreed on that. It's pretty damn impressive how much current a lot of server grade cooling fans actually draw. Even the little 40mm jobbies in 1U servers can pull north of a couple amps at full power.

[@motor-head]

I really enjoy your collaborations with Wendell. If you've never watched level 1 news, I highly recommend it. Informative and entertaining.

[@JamieStuff]

I love seeing the two of you together. Also, Steve's putting down the GPU when Wendell said the price of it was, well, priceless...

[@langx2638]

As a Data Center Engineer as of recent (3+ months) I’m enjoying this type of content from you guys even more!

[@ambhaiji]

Can you please give us a brief roadmap as to how you reach such a position? Knowledge Path and Career Path?

[@runforestrunfpv4354]

@@ambhaiji experience and connections. Comptia server+ helps I heard.

[@agenericaccount3935]

I’m hard coded to play the L1T theme songs in my head when Wendell talks.

[@tanmaypanadi1414]

I just randomly catch myself humming the tune .

[@dllsmartphone3214]

My favorites, Gamers Nexus + Level1Techs in one Video, I love it!

[@whiskeylinux]

Awesome, love both of you guys. Great content as always!

[@lorenzmende]

Some time ago I had a junior tech job at a company building super computers. Supermicro was one of the hardware manufacturer we worked with. The run for dense systems was insane and more than often we had to find solution for the enormous power consumption and power distribution. Another big point is initial power consumption on startup. If you have a rack of those beasts and all systems are startet synchronized, it most likely leads to a flying safety fuse. The supermicro systems are designed really well and the manufacturing is on another level. I met systems scoring a top 100 and better, systems having more memory than my own pc had disk capacity, single CPUs which cost more than my car and insanely high throughput network (Mellanox 56Gbps at that time). It was a great time for me, learning a lot each day and having a deep insight on this field. Thanks for showing the system and creating a nostalgia moment for me.

[@slartibartfast2649]

I just came from the LTT video. I don't seem to be able to escape Wendell. Thanks for the great video!

[@hotstovejer]

Wendell travels via telecom rack. You can't escape him.

[@tardvandecluntproductions1278]

He's inside your pc

[@popcorny007]

Could you link the LTT video with Wendell? Searching for "LTT Wendell" or "Linus Tech Tips Wendell" comes up blank, and their thumbnails/titles don't help either.

[@slartibartfast2649]

@Grey-man m.kzbin.info/www/bejne/sJTGmXerbLCij9U It's not a collab really, they just use Wendell's Threadripper workstation for a video and give him a shout out.

[@popcorny007]

@@slartibartfast2649 Thank you!

[@aznwierdone]

Steve's reaction to handling high cost parts: 16:22 Linus' reaction to high cost parts: *tosses card around*

[@scarletspidernz]

Linus has expensive insurance Rest of us would treat the hardware like Steve 😂

[@YeOldeTraveller]

Steve has to pay for his toys. Linus does not.

[@vladyslavkotov7570]

collabs with Wendell are fantastic, he's something else)

[@sumikomei]

I love Wendell so much, such awesome company to be around.

[@desertmike680]

I use to be a Data Center Engineer for HPE - Absolutely loved working on ProLiants, Blades, Synergy's and Apollo based servers. This video brings back good memories.

[@TheDementation]

I worked in a banking datacentre for 6 years. We has about 1500 servers like this. One build of 150 servers had a $5 million build price. The internet banking servers were a 4 cluster array, each one a quad CPU board with 256GB of memory, the 4th cluster was a hot swap cluster, and the entire thing was mirrored at another data centre.

[@ans5932]

damn, good to see Wendell again, i've been off the loop of tech news for some years now but it was always a pleasure to hear him, and amazing to hear him again

[@bits2646]

Is this my day? I was just doing some deep dive into ROCm, and there it is Wendell with MI210 !! And when he said "multiple videos"... Music to my ears, my heart just started singing :DD

[@antontaylor4530]

This thing is exactly why Intel is trying to get into GPU's. And exactly why they won't *ever* ditch the GPU market.

[@TheNerd]

yep, theres a lot of money in data centers.

[@DrakkarCalethiel]

The ammount of engineering that went into that server is just mindblowing. It's insane how much computing and power you can fit into such a tiny space if you have the talent, money and time. Really wanna see a teardown of one of those MCM GPUs in the future.

[@oragamiYoda]

What a great collab two of the best tech channels

[@Tr3xShad]

"We don't do that here" 😂😂

[@DataQueue]

Wedel is always such a win. scary what you could do with a few of these, let alone hundreds

[@LtdJorge]

Run Crysis on CPU on a cluster

[@agentb4074]

I know absolutely nothing about servers and supercomputers. Honestly, when I saw the title and thumbnail, I thought I probably wouldn't care about this video because I'm only into home PCs... I'm happy I was wrong and that I watched this. I was glued to my screen and in awe of the density and perfect engineering of this - and all of it cooled by 4 fans! I can't wrap my head around what I just watched! Also, I loved Steve's shifty eyes when Wendell mentions the price of the Instinct card. 😆

[@DasTechnomancer]

Glad to see the Wendell collab, hope we see more in the future.

[@BWpepperr]

The humor is getting better and better! That intro and the acting/timing!

[@heytheresakitty]

Watched that intro over and over, love it. Expression says it all

[@grumpyratt2163]

Ok I understood about 30% of what was going on but that's why I love Steve' and GN content. Wendell is awesome too love his channel

[@TalesOfWar]

Don't worry, even Steve looked confused at some of this haha!

[@Alphax45]

5:45 - The sticker says ATI :) Makes my Canadian heart smile

[@louisharkna9464]

Oatmeal indeed, but it's got cinnamon & maple syrup! I just wish we could get the Bar-Talk between Steve, Wendell, Gordon, and Ian after a few drinks... Or even better, a podcast!

[@CirynthLighthammer]

Loved the video Steve, Wendell and GN! I work in a Data Center each day working as a Network Architect and seeing stuff like this is always awesome! And yeah, those rooms get EXTREMELY loud!

[@DtEarth1]

Always great to see a Wendell Level1Tech colab

[@pkt1213]

I see Steve and Wendell, I automatically smash like.

[@darreno1450]

Yep, that's a good team.

[@Bob-of-Zoid]

Thanks for having my buddy Wendell on! Besides being an ultra tech wiz, he does the best Fozzi Bear looks!

[@eldibs]

Wendell is always an amazing guest star, he could have a whole career just showing up in other tech KZbinrs' videos, and yet he has his fantastic channel to boot.

[@whoevertf]

"It's appealing." Goddamn it, I love this guy so much.

[@electrofreak0]

I've been watching some of the older vids from 2 years ago when you were pulling apart servers with Wendell. Super happy to see a new one. Love teardowns of components and devices that I, as a regular consumer, probably will never see in real life.

[@domhenn2006]

I love the Wendell and Steve collaborations. Hopefully more of them in the future!

[@Dr.RichardBanks]

You *_HAVE_* to use that sound bite of Wendell yelling when you're doing fan review methodology or anything fan review related.

[@Meichie228]

Nice to see Wendell as guest on GN. Can't wait for him to get this server running.

[@dwlw668]

Thank You Gamers Nexus, Nice to see a SkyNet Box!

[@markmulder996]

It's always fun to see guys like Steve and Linus coming to Wendell when things get really complicated.

[@Some_Guy_From_The_Internet]

Nice to see you here Wendell! Logan also looks good with his hair grown back up!

[@andreassheriff]

There's so much power in that server it's warping the mod mat. lol

[@xenthoxin]

Yay Wendell! Love seeing him taking these trips to everyone.

[@kevlarandchrome]

I love that the most advanced stuff has gone back to basically a mini version of ribbon cable.

[@DarioCastellarin]

AWS hardware engineer here: I can confirm this server requires hearing protection to operate around, especially if you have a full rack of these...

[@mikepetersen2927]

We loves us some Wendell! Always a fun time when he brings his toys for show & tell.

[@stevo948]

Wendell had soft and steady hands, perfect for handling this sort of thing

[@nukedathlonman]

Level1Techs is such an underrated channel. :-)

[@Christianrosenhoj]

There are currently very few data centers able to power an entire rack of compute dense nodes. With the meteoric rise of computing power since 2015, data centers are not able to keep up with power and cooling demands. Realistically, a standard 42U rack 'can' fit 21 of these 2U nodes but will rarely do so. One, top-of-rack connectivity is still needed. Two, the vast majority of high end data centers have a power limit of 30kW per rack (even the case for some of the largest ISP's). This means that most racks will only be half-populated with these dense configurations. Also, focus is shifting towards liquid cooling in data centers as it is not feasible, realistic or price-effective to keep jamming hotter and denser systems in one rack. This is why many server manufacturers are coming out with liquid cooling-ready servers. These dense configurations are mainly popular with hyperscalers and companies with a massive AI/ML/HPC footprint who are capable of cooling it. Not being able to cool it properly carries a significant performance penalty. The vast majority still look at performance/$ and will prefer lower specced hardware that is easier to power and cool. A topic that might be interesting to cover is multi-node connectivity. These compute clusters are so latency sensitive that once you are scaling out (let's say more than four nodes in a cluster), you cannot rely on Ethernet connectivity. Ethernet wants to deliver the correct packet instead of delivering the packet as quickly as possible, and can result in up to 30% performance penalty. Which is where Mellanox Infiniband (=Nvidia) or Intel OmniPath come into play (=Cornelis Networks). That 100Gb card looks like a Mellanox. The OCP cards are also pretty cool, essentially a standardized format across server vendors.

[@dnakatomiuk]

Don't about anyone else but it's always nice to reveal brand new fresh hardware, when I built my new rig 2 years ago which was my first in 10 years. There's nothing like the smell of new hardware of the knowning of new hardware when building

[@avedominusnox6135]

Wendell!!! Always great when he is on.

[@DanjaBis]

As someone who worked in IBM p-series servers, it's fun watching new server stuff. I kinda miss building racks of them.

[@TheKazragore]

When _Wendell_ says it's the most powerful computer he's ever worked with, you know it's serious stuff.

[@bernds6587]

for that price it better be

[@cubancoffee]

I can listen to Wendell server talk all day even though I have not nor will use this grade of equipment.

[@phlogistanjones2722]

I love your dynamic SOOOO much... Thank you Tech Jesus and WENDELL! :)

[@veleriphon]

It's comforting to know that Wendel is a proud punster.

[@Vultrux]

This video was just great. Thanks for sharing with us and having Wendell on showing us just how awesome this thing really is.

[@RandomBSOD]

i know this is not GNs cup of tea but I absolutely love videos of hardware like these 🤘

[@MrMcGreed]

This is perhaps the greatest collab on YT since GN and Jay did the XOC-"competition"... holy hell, this is pure awesome!

[@andljoy]

Customer: We need more density Dell/HP/Lenovo: Its not possible you cannot break the laws of physics Supermicro: Its true you cannot break the laws of physics ...... but you can bend them .

[@YeOldeTraveller]

In fairness, they are solving a different problem. In most data centers these days, the ultimate issue is heat. When you can already max out the heat budget for a rack, making the system denser is not a win. Also, there is a service cost to higher density. HPE also sells support, and the higher density systems require more effort if the technician needs to replace something.

[@therealsehrlich]

Ngl, I hit the thumbs up as soon as I saw Steve's expression while Wendell was peeling the plastic. 😂

[@drunksupportcharacter]

I do find it sweet that i see this stuff on GN, ill probably never get into server stuff, but just getting a general understanding of that world its pretty interesting.

[@marion_roberts]

Cool to see wendell. Last time I watched him was way back during tek syndicate.

[@nikolaykrustev3679]

Steve is so excited and terrified at the same time. Luv it! Great vid

[@vensroofcat6415]

I will take two. For heating this winter :) Thumbs up for all those collabs lately.

[@kelownatechkid]

Always good to see L1T x GN videos!

[@yunder.]

That opener LOL

[@Matt-bd4qx]

I'm in awe of the specs of that machine. Its crazy how in 10 years this video will go from jaw dropping to funny bc of new technology

[@terbo2000]

Wendell Wilson of Level 1 Techs on Gamers Nexus? What is this a crossover episode? You love to see it.

[@joelcarson4602]

If there's something strange, In your server room, Who you gonna call? CALL WENDELL!!

[@arthurmoore9488]

But, what if it's a Wendel that appeared and is strange?

[@heretustay]

what a truly lovely video. you two have great chemistry and that server was INSANE

[@dougler500]

Yes more Level 1 + GN videos!

[@idkthetime]

Always happy to see Wendell on GN, he always got something cool to share!

[@jannegrey593]

It takes guts to even approach it. I would be scared of breaking it by accident. VERY efficient use of space and a lot of engineering going to remove the heat from the server. Wow

[@CCISolitude]

Two of the most techie tech-bros on YT at the same time, gotta love it!

[@__--JY-Moe--__]

bravo! what an incredible machine! I can't wait to fire up those FP32/64 systems!! kudos to the crew!! so exciting, what the future holds in store! good luck !

[@AgentLokVokun]

Somebody went crazy making this hyper optimized with space AND thermals.

[@runforestrunfpv4354]

These servers are stacked on top of each other on a rack.

[@prax3956]

its really hard for me to wrap my head around this system... absolutely bananas

[@johngermain5146]

Great Content, and always good seeing Wendell and his servers

[@timearl1524]

Great to see my two favourite tech channels coming together to make content, keep up the good work!

[@Dan-Simms]

Wendell is amazing, I can't wait to see move over at lvl1

[@Ravenousjoe]

Wendell is a gem that needs to be protected at all costs!

[@motecasa]

Thank you for inducing me to Wendell, I will now go subscribe to his channel

[@DaleEarnhardtsSeatbelt]

Big fan of L1T and GN. This is great. Looking forward to the server build series! also, ASUS has toolless m.2's!