How To Save $$$ Powering Your Servers

[@JeffGeerling]

Jeff's guide to lower power consumption: replace each server with a Raspberry Pi ;) In all seriousness, I'm glad power efficiency and per-watt performance numbers are gaining popularity. Especially if you consider running these servers in a homelab...

[@hannesstefani]

Hi Jeff, im running 14pis in my homelab. They consume over 200watts😅 my other servers are shortrack supermicro x10sdv-tp8f with 4/8t and a total power consuption of 30-40watt each. But heck yeah Homlab is fun :)

[@ServeTheHomeVideo]

1st? You are totally correct on this, yet it is something we rarely see discussed at this level of detail.

[@JeffGeerling]

@@ServeTheHomeVideo I just refresh KZbin all day waiting for your next upload :D

[@aimannorazman7959]

@@hannesstefani embedded x86_64 is awesome. They just tick all boxes, eventhough they are a little on the pricey side.

[@PatrickKennedy2]

@@aimannorazman7959 stay tuned to STH this week

[@remainsmemories626]

I work at HPC sphere, and I can tell you from my experience: going from air cooling to water cooling not only decreases your temperatures by A LOT, but also cuts your power usage by as much as 30%. When using water cooling (without refrigeration), you use at most 3-4% of your power consumption to cool your servers (typical consumption is around 1-3%, that means to cool 100kw of load you use 1-3kw of additional power). When using same servers with air cooling not only you can fit less in your rack, you also need 20-30% more power to operate those loud powerful fans. (We did tests, and when using air cooling under full load one of our test servers was using 1kw total power, however when using water cooling total power consumption dropped to about 700-750w, up to 720 of which were dedicated to CPUs)

[@ServeTheHomeVideo]

Totally true. We did this with 8x NVIDIA A100 systems last year as well kzbin.info/www/bejne/an_TYnukjM57mJY

[@christopherjackson2157]

It would be cool to see the sth test lab....

[@ServeTheHomeVideo]

It is a bit hard. First - we have a LOT of unreleased hardware. Second - right now we have four locations we are using just to get different power, cooling, and accessibility. That will hopefully change in a few months with the new facility, but we had to do this to de-risk the Austin move.

[@DrivingWithJake]

Awesome to see, we have been working on tests over the past year for all of this in our data center for a blog post.

[@jfkastner]

Well done! When you lower the top GHz speeds by 20% you get 50% less power draw, but since Silicon is more expensive than power most CPU & GPU are run as fast as possible ...

[@ServeTheHomeVideo]

That is going to continue in the near future and so I wanted to focus on some of the other aspects.

[@nickway_]

@@ServeTheHomeVideo Did you ever make the video talking about the other aspects like CPU and idle power efficiency? I looked but couldn't find it.

[@pkt1213]

Really great video. My first home lab was an old Mandiant 1U who's fans probably pulled more power than my current homelab server.

[@Jsteeezz]

same reason I run a 1000 watt 2021 corsair power supply with my gaming computer with a 3090. That 3090 can draw 400-500 watts alone when gaming. Add in rest of system and I can hit like 500-650 watts while gaming. Being in that 50-60% utilization curve gets me the most efficiency and its also better for the longevity of the power supply. Psu’s will last a lot longer if they spent the majority of their life running at 50-60% load versus at 100% load. Worth the extra money to have a power supply I know will last me many years. I know im talking about a gaming pc, but it pays for itself in efficiency and I know I can use that same power supply in a new build 10 years from now.

[@behold_band]

This is why your channel provides so much value. When it comes to large infrastructure rollouts for clients; being able to tell them hey listen, with a less dense configuration you'll be saving X amount in energy costs. For instance with your TinyMiniMicro series, if your replacing thousands of workstations, and the client has these old outdated power supplies. These new, less demanding devices are a huge selling point. Telling a Fortune 500 or comparable company hey, 'I can save you roughly 200,000$ a year with this three quarter million rollout.' They will definitely look past the "if it ain't broke" excuse of not staying up to date with the recent advancements and more efficient devices. From a workstation that is pulling 120w on average, to implement a change of infrastructure to something like the 45w TinyMiniMicro; that can be huge. I live in Las Vegas, and there are thousands of small workstations deployed all over the floors of casinos. They're hogs. I've fixed and worked on them myself. Even in other brands with thousands of workstations nationwide these small savings add up and you bet they pay for themselves by being more efficient. The work you are doing is great. With all the bloat on youtube, you drive home a lot of practicality in the IT sector and it's refreshing to find a channel like this. I've been brought back up-to-date with all the content you provide and badass gear you're able to obtain. Also, I'd love to see some of those big ass 100gig switches pushed to their limits and see if maybe the HP doesn't quite live up to it's competitors in-terms of full-bore field work, etc... In terms of cloud providers, those milliseconds count. Wordpress is a pretty basic work-load. But I also don't expect to run some of these 1U, 2U servers with some Prime95 burn-in. But in the future if you want to demonstrate "full-load" on the CPU and get those fans spinning up to max speed; it would demonstrate a very large portion of potential draw from the PSU. However I think a quick and dirty way to demonstrate CPU/Chassis/Chipset temps would be to do something of the latter. You do a great job of providing information in a manner that is palpable to just about anyone. If I were in sales, I wouldn't second guess sending a client your video to drive home an important upgrade to a client of any size.

[@NamesGolden]

I would like to see STH do some videos on "grid as backup" labs. someone willing to throw $3k at it would be playing on easy mode. throw three used panels on a shed, choose a nice inverter/charger like a victron 24 and snag a lithium pack. to everyone else its "enough solar to run the fridge/sump pump/lights" but to you its also "homelab without a monthly bill" inverter will have a contact you can use to signal a graceful shutdown, so if you only discharge 30% and shutdown non-critical items after five minutes of grid loss you should have critical home loads handled easily.

[@ServeTheHomeVideo]

I was actually looking at big battery backups today. Then I realized we are going to have a HUGE piece tomorrow so I needed to work on that.

[@NamesGolden]

@@ServeTheHomeVideo see youtuber lithiumsolar -- his video 239iu_MsU-A shows a AiO inverter that makes this task super easy. there's all kinds of ways to do it but that's my "toyota corolla / hp elitebook 840" option.

[@gannas42]

The PSU observation makes sense, since the power supply operates more efficiently - a better power factor - at closer to its maximum load. Back when the HP G5 series servers came out our sales engineer commented that the sweet spot is above 80% load.

[@ServeTheHomeVideo]

That is also why often using a single PSU is more efficient than redundant PSUs in a server. We accept the loss to get redundancy.

[@gannas42]

@@ServeTheHomeVideo True point there. Even when you switch the power configuration to be active/standby the system still has to pull some load through the standby PSU to keep it in a state where it is ready to take over. So it can be really important to evaluate whether you need live redundancy or instead tolerate a potential outage but keep that second PSU on the shelf as a cold spare.

[@nekomakhea9440]

It might sound stupid, but a baker's bun rack full of laptops is a fairly power efficient way to get lots of homelab compute for relatively low power and noise, since they are more optimized for those goals than a rack server. A sub-$200 used laptop that's a 4-5 years old from ebay, a pawnshop, a flea market, etc, has built in keyboard, video, mouse, GPU, an SSD or two, gigabit networking, battery backup power, decent IO, and has x86 software compatibility. A Raspberry Pi, if you can find them in stock anywhere, with all the aftermarket hardware needed to add those features, is likely to cost more than that; the only thing you miss out on is GPIO pins, but there are USB GPIO dongles for that. With 3 cheapo used laptops, you can have a OpenStack cluster for under $1000.

[@diavuno3835]

Very cool video! Thanks Patrick!

[@d00dEEE]

I always put these numbers in terms of pay raise when justifying it to the bean counters: "Do you want a 2% or a 7% raise this year?" Yeah, I thought so...

[@wewillrockyou1986]

Gotta love intel sponsoring this when they are worse efficiency wise than epyc in most cases

[@JonathanSwiftUK]

You missed C-states - probably the most important. HPE iLOs have maximum-performance and power-saving modes and the OS has performance tuning. These should save the most. I have a couple of DL385G6 and when I set power-saving in the iLO you could see in task manager that under idle the machine shut off most of the CPUs to save power. Also, for Azure we shut down and deallocate lots of VMs overnight or at the weekend, when they are not needed - primarily to save compute costs but saving power also. C-states, I believe, is where most savings can be obtained, and for most people. One thing I have not seen is a scheduler where you say I want certain C-states during the day, and different ones at night. Why? Gamers talk about disabling C-states, so for them they may be feeling a difference, but for servers you may not notice, but scheduling the C-state level would eliminate any concerns.

[@PhiloSage]

Some of the graphs have extremely small text for the labels. Very hard to read on a mobile device. Lots of white space to increase the text size, would you consider using larger text?

[@ServeTheHomeVideo]

Good suggestion

[@kenzieduckmoo]

When you mentioned 1.6kW power supply, made me immediately wonder if you were running it off the 220-240 watt line cause it’s not likely to break 1.5kW on the standard 110-120 watt outlets

[@DavidCiani]

Not on a 120V 15 amp circuit, but a 20 amp would be able to do it, and they are somewhat common in residential/office applications. That would be ok for a workstation or a single server on a circuit, but no dice for more than one, let alone a rack situation.

[@AmnesiaPhotography]

Now all we need to do is get the people using interpreted languages to use compiled ones or save power.

[@Tech_NO_Tech]

Have you tried using only one power supply instead of dual power supply ? I know it makes a difference on the UPS so the power consumption is reduce by a lot without using only one CPU.

[@ServeTheHomeVideo]

It used to be a lot more. I think we tested this years ago, but with modern servers generally at 80Plus Platinum or higher, and power consumption going up overall, they end up sitting above 25% utilization in most servers even if the power is split evenly across both of them. Some servers still do one main PSU then a very low power state second PSU while running.

[@darren429858]

I am curious when having a 2 socket system how much power is saved between going with a single cpu vs dual, so buying a higher spec xeon or going with dual low spec xeons with the same total core count. I just bought a barebones r640 and am debating what CPU('s) to toss in it.

[@ServeTheHomeVideo]

Oftentimes single is lower power

[@miff227]

I wondered if fully submerged racks ever became a thing with say Novec-7100 fluid?

[@ServeTheHomeVideo]

Yes. Immersion cooling is a hot topic today. We have covered it several times on STH if you check the main site and our liquid cooling videos

[@AndreiNeacsu]

Muahahaha! (Laughs in European at 240V and tri-phase PDUs feeding from 380V)

[@brookerobertson2951]

I have a solar powered calculator so I don't have to worry about my carbon footprint now. I just burn all my trash guilt-free. The money I have saved having to pay to dispose of my old car tyres offset the price of the solar powered calculator.

[@chrisdash83]

what about compatibility? Can you use an Intel server psu in a Supermicro server?

[@karim1485]

Not a server user here, but why do fans consume this much? Can you not attach much weaker fans directly to heatsinks and add bigger heatsinks to the parts that need more than convection cooling?

[@ServeTheHomeVideo]

In servers, you are limited a bit by the density because you have 32 DIMM slots on either side of the CPUs. On the direct attach heatsinks this is usually not done in servers since that is not a redundant cooling setup in the event of a fan failure

[@caprature]

Wouldn't different variations of ram configurations make a big difference in over all impact. Like per say 256gb across 8 DIMMS vs 16 in the same system.

[@ServeTheHomeVideo]

They do make a difference, but it is often a lot less than some people think for a lot of lower-density applications. Now, Optane on the other hand can add quite a bit of power.

[@caprature]

@@ServeTheHomeVideo thank you. Love the content you guys produce. It’s been really helpful getting back into the enterprise market after a 5 year hiatus for myself

[@HikaruAkitsuki]

Can you experiment what is the saftest way to install highend graphics card on 2U server like Dell poweredge r710?

[@victorcoss2600]

I've talked to people regarding 120V vs 240V in a homelab, as you know 208V isn't normally available in a residential home unless you have 3-phase power ran, good luck with that.. While 240V is more efficient, in a home lab 120V is adequate because UPS's for 240V are gonna be a lot more expensive and harder to come by, and in a homelab you aren't scaling out thousands of servers to make that big of an impact, it will be negligible.

[@ServeTheHomeVideo]

I actually have had an electrician out to scope putting 3-phase power to my new place. :-)

[@victorcoss2600]

@@ServeTheHomeVideo There are plenty of places where you can't even get 3-phase power, there is only single phase down my road. Even if I were to get it, you'd have to have a separate electric panel, as the majority of stuff we use in a residential setting is 120V. I heard 3-phase power is only really necessary for 3 phase motors. I'd love to know what they would cost you, because you'll have to have a new transformer and everything.

[@berndeckenfels]

1% difference, now that would have a multi digit error margin by the PTCs, air pressure, ambience and different fan vendors/batches … the question when comparing 1u against 2U you of course need to compare 2x 1u

[@Claude-R1]

Switching to AMD literally improve efficiency by 30% instantly

[@ServeTheHomeVideo]

This is a bit of an older video, but now with Sierra Forest Xeon 6 and Intel at 144 cores in 250W Intel is now beating even some Arm server CPUs in efficiency and has more cores than AMD even at low power.

[@gamotronix_sakib6548]

I don't understand a word you say. Then Why do i watch these videos 🤣🤣🤣🤣 Please make a video What is al this that you talk about

[@wcvp]

18:27 shouldn't that say "megatrand" instead :P

[@SB-pf5rc]

noobie here- what's the difference between a 1u and 2u server?

[@АбракадабраКобра259]

1u very slim server, only uses one unit out of available rack units. in dc it's usually 42u or above that in one rack, 2u will be a bit taller and will use two out of those.

[@selfelements8037]

You are getting a lot of Intel sponsorship lately. Don't forget the better and more reliable company AMD.

[@ServeTheHomeVideo]

We do a lot with AMD and others too. Heck, check Twitter today :-) twitter.com/Patrick1Kennedy/status/1496142675136892934 Also, the TMM series is going to be very heavy with Ryzen so usually I try to balance out everyone's coverage on STH. Sometimes that is easier than others.

[@selfelements8037]

@@ServeTheHomeVideo Yes, I know. And you also talk very good things about AMD in your channel, which is positive and true. Don't change! (:

[@JViz]

I know you have to make money, but plugging CPU accelerators is definitely in that morally ambiguous territory. I know well enough to not touch them but a lot of your viewers might not and you're kind of banking on that. Maybe if you touched on the reasons not to use them it wouldn't be so bad, but I doubt you'd keep your sponsorship if you did.

[@ServeTheHomeVideo]

That is an interesting take. So you are not using AES-NI or other crypto accelerators on the server or client side? The wide adoption of crypto accelerators has allowed the Internet to adopt HTTPS, VPNs, and so forth because it cut the overhead by an enormous amount. Putting it more succinctly: almost every web page you see uses CPU accelerators added 10-12 years ago that are now standard in the industry.

[@DanangAlta]

Dude talks about power consumption but negates the fact that all of the internal components run on DC current and does not factor in DC as the source AS OPPOSED TO AC in any of his efficiency test, instead, he goes to 220V AC, which makes no sense at all. People use solar to power these things, that POINT was missed too. Dell also has in their SKEW a handful of DC to DC PSU's, but no, this dude did not even bother to investigate that aspect either.

[@inexpensive_id]

hope *you* *guys* *are* *intertained* *with* *my* "penyelam amatiran" *video* . *happy* *watching* *and* *enjoy* 🙂. *Passion* *for* *work* 🔥.