Shiny New 128 Core DevOps Build Server For Greg Kroah-Hartman! AMD Epyc ftw

[@_MrSnrub]

Wendell, you have come such a long way. You seem very comfortable on camera these days, and I really enjoy watching you divulge your knowledge to us mere mortals.

[@edplat2367]

Yeah, he used to do tech content and now he just shills whatever product sponsors him.

[@cloudstrife7083]

what he wasn't like that before ?

[@edplat2367]

@Dominic Cerisano I can understand people need to get funds from somewhere but honestly I am sick of just seeing builds with sponsored hardware. You do realise they used to make much more informative content. They've covered a lot of brands content saying only positive things about them with none of the negatives e.g. asrock.

[@zomberton616]

@@edplat2367just leave then

[@Banner1986]

@@edplat2367 asrock rack's hw is pretty damned good, but I dont really use any of their consumer stuff. I feel like the world wendell comes from is similar - while many of the consumer side products have various issues, I find it pretty easy to gloss over them as those issues can often be attributed to using the motherboard for some server-like workload. Gigabyte is the one I have problems with, at least personally. Shit support behind any product, no matter how solid it is, makes a shit company. I'll never pick up another server or consumer component from them. I think L1 actually noted some concerns with them as well, but that's neither here nor there at this point.

[@Maxjoker98]

I like the charity aspects. Keep in mind that Linux literately runs the world, and every hour GKH isn't sitting in front of a computer doing is freeing up some of the most valuable developer time on this earth. So in whole, this probably donated way more than the hardware cost to the Linux project.

[@llortaton2834]

Wendell, just your friendly computer janitor, with a LOT of keys on his set and knowledge.

[@Deveyus]

I pay attention to the companies that said "Yes" to this stuff. That'll be the tie breaker if I have to choose between any of them.

[@raven4k998]

imagine what a 256 core cpu would be like cause oh yeah it's coming I can feel it

[@Nalianna]

Same (and well, EVGA forever) :) I'd never heard of this brand of PSU though... i love the idea of it, except... not available in my country. my NAS needs one :)

[@llortaton2834]

i love that atx sized double PSU, i use it on all my 4U servers

[@NdxtremePro]

Great vid, but you forgot to shoutout to the SSD folks at the end. Hope they don't get mad, thanks Kloxia.

[@wyattarich]

Another fantastic video, always exceeding expectations (which are getting pretty high). I'm glad it's all going to such a good cause again! GKH FTW

[@junkerzn7312]

Still loving my 3990X workhorse! It just chugs away, day in and day out, without complaint. I don't like putting those huge tower heat sinks on their sides, though. That's gotta torque the socket... not a 10-year plan IMHO. I always build them so gravity is assisting in the right direction, even if it means using a 3U or 4U case for the larger systems. For the VRMs, I always offset the intake fan on the tower cooler down towards the motherboard as much as possible, so a small part of the fan is actually pushing air under the cooler instead of through the cooler. That seems to solve the problem for me. -Matt

[@tanmaypanadi1414]

Thank you for sharing that fan placement tip.

[@DarkShadowReign]

I do the same thing with my fans on my Noctua NH-D15

[@cldpt]

The board and backplate are taking most of the load, then it's torqued to spec. Buildzoid had some thoughts on pinout loads and apparently even Intel 1700 could take tens, almost hundreds of KG of pressure over all the pins it uses. TR4/SP3 have even more pins over a larger area. I also wouldn't worry much about stress on the chip itself if they state in the spec it's supposed to be pressed to such extents on the socket.

[@ryanb7476]

@@cldpt agreed. Ive had an NHD-15 on one of my systems for 8 years now, its fine and it hangs big time. same on my current threadripper, high end boards arnt made of thin paper, they got back plates etc, plus the included toque wrench on threadripper insures its all correct. nothing to worry about.

[@elikirkwood4580]

Man what a badass setup. Im also really glad to see a good air cooler for that socket that points the fans correctly for server cases. that was one of my concerns with going with an epyc setup in my server.

[@PhoeniXfromNL]

lol loved the ending, yet as a gentoo enthousiastic since 2005, that box just seems awesome all the way, am sure GKH gonna put it too some good use ^^

[@TommyApel]

yeah, makes me wonder how long it would take from chroot to desktop login if you stuffed it with 1tb ram and used half of that for a portage ramdisk for building.

[@YouTuber-jz5nd]

I have a Torrent case and I love it. With an AMD 7950X, a PowerColor 7900XTX, 64GB of DDR5-6000 CL30 RAM, and a Noctua D15 cooler. With the included fans the thing just stays so cool and quiet, it's great. Also, a solid metal side and no RGB, how great is that!

[@Flojer0]

This feels cool to see while currently owning a machine that excels at everything I do on it. Nothing like this, but tools keeping out of the way feels amazing.

[@benpope8706]

Wendell, you tha man! When I first watched your early videos I KNEW that we would see great things from you! Just imagine all of the kernels compiled on this beast. Amazing build!!!! Awesome video!!!!!!

[@nuttywombat]

Going to interview the commander in chief again? First one on the topic of computers barely working was awesome, and would love to see you guys geek out over Linux and hardware again.

[@tyler5888]

Totally. Not to mention, Level 1 providing ram and coolers to help GKH is basically directly assisting the entire Linux community so no harm no foul there.

[@ohkay8939]

So cool and infectious to see Wendell geeking out. Awesome video, thanks for sharing :)

[@johnpaulsen1849]

Love this advice. Define your needs then tailor the solution to those needs.... If only sales would understand that!

[@twistidclowns]

Wouldn't be a level1 video without the boiler snake😊

[@DocNo27]

RE: The special standoff - in the days of nylon standoffs that slotted into grooves in the case to hold the motherboard you could just snip off the part of the standoff that would go into the groove to support the motherboard. Everything new is old again :) Nice build! That dual power supply is sweet - I can see uses for that. I will never build a system like this but still fun to live through others. If we had KZbin in the early 90's would it have been as much fun? I wonder...

[@TheBoltcranck]

Wendell sharing the experience of building a PC/Server/Workstation that will impact the Linux community is wholesome to watch.

[@marek.lochki]

Wendell, Thank you very much for putting on such a fantastic build video, its definitely the best build video I have seen in a long time. The way you present the information is great to digest.

[@henderstech]

Wow what a build. I can only dream of such hardware.

[@christophermcdonald5578]

The power supply form factor is pretty slick

[@sheldonkupa9120]

Sick, how entertaining and relaxing to see you at work!😅👍

[@javierortiz82]

I'm in love with that chasis, and there's a version of it that doesn't have a glass panel!!!

[@FutureChaosTV]

I love the Torrent. I was about to order it when it was recalled because of the fan hub issue. It is sooo well thought out and built. My previous Corsair is also huge and not bad but when I built up the Torrent case everything just made sense. The only "gripe" I have with it that it could have one centimeter more space on the side behind the motherboard because of the amount of cables I have running through there and the SSD mounts are a bit iffy as I have old SSD's with the screw holes on the "wrong" side. No glass side for me either. I love it!

[@javierortiz82]

@@FutureChaosTV it's a shame it's not available in my country, it would be very expensive as well.

[@dmdx86]

I looked into the FSP Twins for my ATX server build but saw they chose to use a USB interface /w proprietary Windows software for monitoring instead of PMBus like every server board has.

[@ReinaldoRauch]

Awesome vid! Great support for the linux community. Just one thing that i haven't understood, where the storage went?, Sure, Wendel mentioned Kyoxia and his drives but not shure where they went in the build

[@Junior41180]

that has to be some kick ass compile times!

[@lanklaas11326]

I would love to see a video for a build for compiling rust code. Needs threads and clock speed

[@Alphahydro]

Hey Wendell, glad you went with a heatsink with appropriate orientation for the server socket.

[@onkelpit]

Danke!

[@gerowen]

4:48 You've got some kind of "really" deep bass tone that hit right about here for the second time. I think I'm only picking it up because I've got headphones on, but it's kinda weird, and kinda pleasant all at the same time. I think you must have had your camera mounted to or sitting on the table when you plopped that box down onto it and then patted your hand down.

[@jacob_90s]

I would really like to see some performance testing for pfsense. I know that for 99% of situations using cheap off the shelf hardware can even handle a 10gb connection, but it would be really interesting to see when push comes to shove, where the bottlenecks actually are.

[@nathanl6598]

That motherboard is ridiculous. I feel like I just watched you tame a beast.

[@chromerims]

Weery vell done 👍, very Wendell. 7:40 The HMAA4GR7AJR8N-XN dimms are ECC. Was wondering, so I checked. Much appreciation for your generosity and neighborly welcome.

[@SquintyGears]

This powersupply is really cool!

[@bambinone]

Would love to know more about the server config. What tunables did you change? Did you set up a RAM disk for the kernel source code? Are you running the latest kernel with all the new EPYC patches?

[@Level1Techs]

then the other videos mentioned are the videos for you :)

[@bambinone]

@@Level1Techs OK! Thank you!

[@TommyApel]

That's actually a very neat build for an on-prem build server that doesn't pollute the environment with noise.

[@EposVox]

Hot damn

[@ChaJ67]

While I can understand how liquid cooling can be high maintenance by running two liquid cooled computers myself, I have found there are ways to keep the maintenance down. The biggest thing is to use clear coolant. I have been using the clear EKWB coolant for years now, which is a glycol mix with biocide and my flow meter has been measuring the same max flow rate for years now with just a simple yearly flush with the help of an air compressor, proper use of the regulator valve on the air compressor, and a Pyrex bowl with a lid to handle splashing. (The idea being in addition to a well engineered loop that drains well, I use an air compressor to more completely clear out the loop of the old coolant.) When I used coolant that was colored, that food dye or whatever they used just trashed the loop and provided food for the algae to grow. Then I had to get out the toothbrush, long needle brushes, etc and dismantle everything to clean that mess out. I used to hear people say that is what you had to do every year or so when I got into custom liquid cooling loops and with the colored coolant I would agree, but not with the clear coolant I am using now. If you want color in your loop, use LED lights and some colored tubing; don't use colored coolant. But yeah, I would argue a well cooled computer with quality liquid cooling keeping all of the hot running stuff cool (and I am using a monoblock on my workstation board, which is a rarity to find and I had to bug a German guy to make it), it will likely outlast an air cooled setup just because it runs cooler. I suppose that is the other thing, I invested in good sensory and control hardware to keep the cooling system running optimally and cut power in case of a serious cooling fault or if it just gets way too hot in the room. With dual 280W CPUs, that is a whole lot of heat to dump into the case and back to back 120mm coolers is definitely not ideal for that kind of heat output. A high flow rate coolant loop will whisk away that 500+W of heat and barely warm at all between the two CPU water blocks. It would be interesting if you could find a block to stick on those VRMs. I am finding my liquid cooled VRMs with high end pads on them stay rather cool. I have lost hot running VRMs in the past with things like the system just turning off while under sustained full load to a loud bang as a power cap blows out and then the system shuts off with a burnt electronics smell filling the room. So keeping those VRMs cool is important even if they are rated for a high temperature as they don't always last at those temps.

[@virtualinfinity6280]

Actually, I wonder how a pair of the shiny new 7773-X would do in compiles. it's massive L3 cache should help quite noticably. But maybe these chips are unobtainium right now, and BIOS support is at beta currently.

[@guy_autordie]

I'm sure AMD would send these if Wendell asks them. But maybe at a price, even for gkh.

[@PRiMETECHAU]

modprobed-db and localmodconfig helps allot speeding up kernel compile time since you rarely need all the driver files for a specific system. Just gotta keep in mind its not perfect and sometimes misses out modules so you gotta keep a eye on that but I use it myself and it works quite well even though it forgot to include some core filesystem modules.

[@cldpt]

Amazing video! I'm considering one for my company (our first build/proxmox server!) and have some questions: - we're gonna go with single Epyc 7002 Rome and 64GB over 4x16GB, for cost and initial testing, upgradeability in the future - I'm choosing between Supermicro H11DSi (for 7002/3) and this Gigabyte (which has Rev1 and 3),: Supermicro boards have 2 benefits, such as upgradeability to Milan (7002+7003) while Gigabyte needs to stick to Rev1 which is 7001/2 only and Rev3 does MIlan. It also has offset sockets which may help cooling. I'm leaning towards Supermicro but the Gigabyte looks fancier and has full x16 slots. Anything wrong here? It is also slightly cheaper. I think they both will fit ATX fine. - wondering if the Gigabyte Rev1 has firmware upgrades for Milan? What are your thoughts on such a machine for a proxmox doing both build servers and on-demand Windows/Linux remote workstations and LXC containers?

[@Felix-ve9hs]

Afaik FSP also made (or still makes) the be quiet! power supplies

[@andrebalsa203]

Impressive build ! I would have suggested disassembling the heat sinks before shipping but I suppose it's too late now.

[@Jango1989]

Wow! This is absolutely amazing 😍

[@wskinnyodden]

Wendel! Are you secretly building my birthday gift? It's a bit early, August is still months away! Please don't forget to put an R6900 on it, I also want to try some funky GPU acceleration for things it was not really originally designed for :P

[@FuelTim]

I really like those coolers. And great video!

[@MatthewHill]

"You should know exactly what problem it is you're trying to solve, before you ever spend the first dollar." The problem I'm trying to solve is I JUST WANT TO SIT IN CLOSE PROXIMITY TO GOD-LEVEL COMPUTER POWER while I do my mundane office spreadsheet job.

[@rdsii64]

Jeff over at kraft computing tried to put those freezer coolers in a 4u chassis and it was 5mm to tall.

[@morosis82]

Yes, that's what shied me away from them. It's possible they might fit in some 4u chassis, but I measured my Supermicro 743 and it was too close to call. I guess with the money already spent it wouldn't be a big deal to check (Epyc 7452, 256GB DDR4 3200 SK Hynix, 4x U.2 drives for VMs, soon to be video card of some description for cloud gaming).

[@morosis82]

Hey, I just put 8 of those exact DIMMs in my 7452 Epyc homelab server. I got them for a very reasonable price.

[@arwo1143]

For the people wondering about cooling (as I did) After a quick google search: apparently water loops can handle about 100W per 120mm fan, and about 130W per 140mm fan for “reasonable temperatures”. This appears to be the case for your general AIO 27mm radiator EK lists the cooling capacity for one of its own 60mm radiators at up to 480W at 2800 rpm and a ambient/ water temperature delta of 10C With the important note that radiators get more efficient with a greater temperature delta between ambient and the liquid Sooo…. That’s nuts Did not think that would be that much

[@mirrorking2]

No mention of the vrm temps? It's the vrm that gets toasty in dual epyc desktop builds. The CPUs are surprisingly easy to keep cold with the massive dies. But those tiny vrm 6/12 phase heatsinks struggle without massive airflow.

[@aluckyshot]

Love your presentation style and jokes.

[@dariuszszczepanik8110]

What is this dual U.2 PCIe card at 9:40 ? I would like to use it in my build...

[@MarkRiker]

I pressed that like button hard when i saw that "no tempered glass".

[@oakfig]

That case looks awesome

[@evanjrowley]

Hope to see more collabs between L1Techs & GKH!

[@Alexandra-Rex]

I love Arctic's products and didn't know about this cooler. I guess on a normal motherboard it would not blow air horizontally, but vertically.

[@MrChomiq]

Boiler snake wants to run a code compile on this ASAP.

[@garethevans9789]

I'm surprised you didn't mention the lack of sound damping on the panels. The case isn't cheap (you only save $10 over the windowed version). I expected more from Fractal (used the original XL, with some mods in my main PC for over a decade). Also, I had to replace the bottom fans as they were resonating with the case. Making 950-1100rpm unusable. While quieter above that zone, they're not quiet.

[@bitrage.]

Your channel always fills me with GREAT ENVY..... But ur sparkle effect on the thumbnail looks like snow...🤣

[@yokailover]

OK, driving me nuts....what is that handheld light you're using in this vid? I could really use one!

[@bzmrgonz]

Why not add a shroud to the dual heatsink/fan sandwich to direct the hot air out of th ecase??

[@ReQuiem_2099]

Lol, I love watching Level1Techs skits

[@NotAFanMan88]

Now I'm just thinking of this build with the Delta THD2048HT fans that linus showed off.

[@Squash_Man]

I think even Dell stopped doing inline cpus on their dual socket workstations to keep the back socket from overheating.

[@almaredss]

Can I download the Windows system and use it to edit the video, and do you install a screen card 3090 Nvidia on it?

[@Dannicus117]

I like the eco-friendly comment with the nuke the whales background 👌

[@tanmaypanadi1414]

I just noticed that 🤣

[@techfan7808]

Questions - Why did it turn out that dual socket 32 cores turned out to be optimal, as opposed to quad socket 64 core? Does it have to due with parallelization, Thermal Limitations, Fabric between CPUS, Scheduler issues? Or something else?

[@zaneandre6387]

Boiler snake...daphuq? LoL good work buddy.

[@denvera1g1]

Just for schidts and giggles, it would be fun to take some trident Z RGB 4400CL19, and see what a threadripper system like this could do with say, 4000CL18. Also 16 sticks of RGB would be an interesting thing to manage

[@nathanlowery1141]

Probably be hit and miss. Silicone lottery with the memory controller. Doubt you could hit that rated speed with quad rank

[@nathanlowery1141]

Ohh and definitely couldn’t run 1:1 at that speed. Fabric speed is finicky

[@denvera1g1]

@@nathanlowery1141 Oh, yea, only Intel and 7nm AMD APUs can even come close to those frequencies(my 5700G is running 4933CL17-17-17-28 in 1:1:1) But even at 3600 you might be able to drop the timings to CL14, even with 8dimms per processor you might be able to hit 3800CL16

[@carlosc3260]

I came here for the Snake Jazz… didn’t disappoint

[@Dan-Simms]

What a build, the snake just wanted to say goodbye to it.

[@SomeTechGuy666]

Those fans are not compatible with rack mount 4U cases. At least not the cases I tried. They are about 5mm too tall.

[@steveseeger]

Are the CPU cooler fans airflow all one direction? Wouldn't you get the hot air from one feeding the second one making it run hotter? Pulling air from the center and out opposite ends could work but may be tougher to manage air flow or be turbulent.

[@tanmaypanadi1414]

custom acrylic air flow channels like Puget uses should solve the issue. but it might need some thermal testing.

[@a4e69636b]

I was hoping to see more benchmarking.

[@98f5]

FINALLY someone builds an actually amazing system on youtube. not just the same typical shit. zen3 epyc amazing.

[@doxydoxdelamanca9902]

Always wondered what kind of cell phone you use, Wendel. I wonder no more! :)

[@alpenfoxvideo7255]

where is the DUAL Epic watercooled build video?

[@thibaultmol]

I can't find the dedicated video on the fsp psu

[@rdsii64]

I wonder if you can lay the torrent sidways in a rack chassis.

[@muhammetfurkancankaya777]

You are making great videos.

[@anthonyc417]

Should really put a cap on that valve in the back of that custom copper build.

[@shell4331]

What distribution did you use for this outstanding build? Did GKH went with Arch for his build server?

[@doxydoxdelamanca9902]

Nice; Howitzer of a station!

[@CoolBrittany90210]

FYI, this video is in the workstation playlist :)

[@Wesrl]

I see that VOIP phone on the desk when are we going to get a VOIP video

[@b2bb]

Wendell on an iPhone? This truly is the darkest timeline. In all seriousness, very cool to be commissioned by Greg. _engagement_

[@SamuelLudden]

Can someone please explain the "Nuke the whales" reference in the background?

[@slipknottin]

How well does this system run Stellaris? I know you tried it lol

[@tamrix]

Damn I like that power supply.

[@jtrevathan33]

Over on your fleet limit and starbases while running out of alloys 😬

[@Outland9000]

I don't think I have ever actually finished a game of Stellaris. I get part way through a game and think _"What if I had done it this way...?"_

[@jonathonrosalia9345]

for those of us who don't have a lot of moneys is the cheap epyc first and second gens worth it?

[@TeraQuad]

"Gotta Nuke somethin'"

[@jessicantina]

How much did Paradox chip in for the Stellaris cameo is the real question.

[@MrNeocortex]

That's a happy coincidence. I'm also playing Stellaris.

[@coralking9]

What game was he playing in the beginning of the video?

[@SkysTrains]

would that cooler be worth it for a 24 core 7401??

[@MichaelPickles]

from what i have read and seen the cpu coolers will not fit in 4u they stick out by about 2 mm

[@Level1Techs]

Check the level1techs website. Depends on the case. Working fine for me.

[@Dievolve]

sick HPC build!