Server Rack Tour 2021 - Epyc Rome and Ice Lake, OH MY!

[@saulgoodman1390]

For a professional beer review channel, you have a hell of a network setup!!

[@LAWRENCESYSTEMS]

Nice rack! Also, home assistant is a great way to take control of all of your home devices.

[@BoostedNW]

Home assistant is a must for sure! The addons and default features will be all you ever need!

[@nevoyu]

"I want self hosted home automation!" Home Assistant? I've been runnig it on an airgapped system for a few months now. Been a non issue

[@TheRogueBro]

Came down here to say this. TONS of videos on youtube. Heck, even Linus is going to run it in his new house.

[@muddyland6910]

I was thinking the same lol It's amazing how people don't find this stuff. Home Assistant has changed my life! Self-hosted, private, feature rich, amazing community. What else could you ask for?

[@stephenp4440]

I also use Home Assistant with ZWave, Ubiquiti, etc. I have redundant Raspberry Pi 4Bs plus I run it in ProxMox so that I have a local copy on each floor of the house (I have a rack on each floor that has local control and communicates with the others). Home Assistant also automatically controls all of my air cleaners, dehumidifiers, HVAC, exhaust fans, etc. based on environmental measurements all over the house and outside temps/air quality.

[@tjmbv8680]

Yeah I can think of a few off the top of my head, I am setting up an open hab system right now for controlling some lighting in my bedroom, an open sprinkler controller, yard lighting, my home theater system, and my hifi stereo system.

[@HcgRandon]

HIGHLY recommend home assistant, w/ zigbee

[@antoinereese4295]

I'm truly amazed that your not running Home Assistant on all that. Not going to get any better for self-hosted home automation.

[@ConfidentGrips]

Same I think his argument is many of the smart things you can buy require the cloud

[@NathanBorup]

Lol when he started talking about not being able to run a home automation solution locally, I was like bruh... You gotta check out home assistant

[@NathanBorup]

@@testthisfordecficiencies name them?

[@ProffesorFarnsworth]

@@testthisfordecficiencies What are they?

[@jlficken]

I've had that Docker installed on UnRAID for months now but haven't set it up. The updates to it are frequent and huge for some reason.

[@JeffGeerling]

2:35 look at you with your fancy 50 up! Spectrum doesn't go past 40 for love or money 😭

[@CraftComputing]

Oh, I pay handsomely for upgrading from 20 Mbit to 50 Mbit....

[@computersales]

Me just vibing wondering if I should go from $55 a month for 100 by 100 to $70 a month for 500 by 500 🤔

[@sleepysloth847]

Kinda crazy to think that I pay $85 a month for gigabit fiber up and down in central Iowa. Ya'll need to move out here lol

[@andythekitsune]

Rogers won't give higher then 30 up on their gig plans and Bell is taking their sweet time finishing FTTH in this building

[@JeffGeerling]

@@computersales And my bill for 1000/40 is $150/month 😵

[@ProjectswAlex]

"Letting everyone know you are better at typing than they are" CRACKED ME UP! 🤣

[@AlexZanderMuro]

you might still want to consider going with two UPS units when you do decide to upgrade; having A/B redundancy and splitting the load between the two is a great way to not only bring the individual unit load down, but in the event of a failure, PM, battery swap, etc., you have one unit that can remain up and handle the full rack on its own. Plus, if you already have to run a 240 circuit, running a second at the same time is better than having to run it after you get everything buttoned up. that tripplite unit is frickin sick though! i didnt realize they had rack mountable AC units like that. will definitely be looking into one of those.

[@mapesdhs597]

I see a fair few comments about using the vented air to heat the house during winter, but that's the time of year when less such air would be created anyway. Also, as it is, the air would be rather dry, laced with heaven knows what vapours from all the machinery and probably static prone, not something one would want to blow straight into a home. Perhaps a heat exchanger could do something useful, I don't know.

[@Space_Reptile]

"rails are worth their weight in gold" yea sadly sellers for said rails think the same and they are hidiusly expensive for what they are in the end

[@DangoNetwork]

it depends. sometimes just the inner rail, other time just the outer

[@computersales]

Shipping costs are a factor too. Dimensional weight is a pain.

[@kirksteinklauber260]

Nice video!!! Related to Mikrotik RouterOS v7 is in release candidate and this version enables L3 HW offloading so the issue on L3 routing will be addressed and fixed. You may want to explore a video testing this new version!

[@drumaddict89]

be careful with that feature ;) it still is a bit buggy (even on 7.6 as of date writing this comment)

[@kirksteinklauber260]

@@drumaddict89 for me has been working great!! Stable and no issues at all

[@drumaddict89]

@@kirksteinklauber260 glad to heat that. on which hardware you are using it? cheers

[@kirksteinklauber260]

@@drumaddict89 I have deploy it on 5 x CRS326-24G, 1 x CRS312 and 2 x CRS328-24P and all running fine and with lower CPU usage after l3hw-offlod was enabled

[@drumaddict89]

@@kirksteinklauber260 well thank you sir, i need to check that one out on my CRS326 then.

[@attilavidacs24]

Great video. Would have loved for you to talk about the power usage in watts and what is costs to run the setup per month.

[@TheSergeantTaco]

Home Assistant with a Zigbee / Zwave usb stick would be good. Also Home Assistant Amber would be good with a cm4.

[@JeffGeerling]

Agreed. Can't wait to check out Amber!

[@TheSergeantTaco]

@@JeffGeerling holy smokes! I I love watching your videos. Amber seems like a win in my books.

[@UNVIRUSLETALE]

You can host home assistant and buy tasmota flashed (or flash your own) devices to integrate and be completely independent from the cloud

[@miguelgutierrez2759]

The only time we can say "NICE RACK!" without offending someone.

[@CraftComputing]

I had to specifically add "Nice rack" to my comment section whitelist, just for videos like this 😀

[@SpookyLurker]

@@CraftComputing Seriously?

[@mjc0961]

@@SpookyLurker I believe it. It's very easy to imagine KZbin automatically holding comments that say "nice rack" for review because so many scumbags would be posting it inappropriately in the comments of any video with a woman in it.

[@prashanthb6521]

LOL

[@BenCos2018]

@@SpookyLurker what did he said lol

[@black.phoenix.]

I can vouch for the Mikrotik equipment. Last company I worked they had a router board and a switch from them and at the first glance I, who always had used Cisco/Netgear/other enterprise brands dismissed as bad. Yes their management software on the PC is archaic and somewhat confusing, with a somewhat step learning curve. But in terms of hardware for the price/performance bracket, heck they are some powerful little machines. Yes has you said, if you start messing with L3 the cracks start to appear, but again as you said, for L2 that is most of the normal needs of a small to medium business, they just rip and tear.

[@Dgeigerd]

L3 and cracks? Laughs at CCR1036-8G-2S+EM at work! I mean who excepts gigabit+ routing to be good with a SWITCH?!

[@maynardcrow6447]

The chenbro does have a sas controller. Hdd 1-8 I believe are all sas. I have 5 SAS drives in mine BUT you have to use a sata to sas adapter and you won't be able to fit 12 drives with adapters but it does have SAS controller. They might have smaller adapters out there though.

[@GeoffSeeley]

Looking forward to the Supermicro review! The only thing missing is a PDU. With the right PDU you can measure/log input Voltage/Amps, per outlet output V/A, temperature/humidity sensors and the ability to remotely turn off/on outlets of anything in your rack. I have Zabbix monitoring and remotely rebooting a power meter monitoring box via a PDU when it stops responding to API calls (it's old, no longer supported and no direct access). Cheers!

[@SteveSunny]

What PDUs do you have in mind?

[@RubyRoks]

Thanks for the tour. Lots of cool systems, and lots of cleaning to do

[@DocaTafner]

Hello, I found your channel by accident 😅 I'm here to comment and wonder what a huge electricity bill you must have! Still it's very interesting some comments like not leaving your home data center for home automation items. I also love the products from Ubiquity which come very handy to specific scenarios. Anyway cheers, take care!

[@CraftComputing]

Actually I live in one of the cheapest places for electricity in the US. $0.072/kWh. Crazy cheap.

[@DocaTafner]

@@CraftComputing it is indeed! Here where I live there is hydroelectric power generation and it is supposed to be cheap. I was curious and did some calculus and the actual cost is double that, at least! Thanks for answering the comment! 👍🏻

[@edbouhl3100]

@@CraftComputing Ah, cheap hydropower! As long as the rains come, that is.

[@azurite2926]

I’m surprised you still use the UDM Pro. I stopped using anything Ubiquiti related due to the constant bugs and issues. Switched to Untangle which is absolutely fantastic, switches and APs are now Engenius (also working out great).

[@berndeckenfels]

Of course you can put your backup server into another room, on another power storage and different flood-level (in your same home)

[@TechnoTim]

Nice tour Jeff!

[@tumtumsback]

Hey Jeff, long time watcher of your KZbin channel. When you got to the USW and mentioned having issues trying to figure out an anti-cloud home automation solution I just felt like I had to reach out personally to say that you can pick my brain about home assistant. I have over 100 devices in my house, from bulbs to switches to sensors to TVs and much more, all with automations galore. I would love nothing more than to give back and help you out if you need any guidance. Oh, and I have a dedicated beer fridge in my basement … Cheers 🍻

[@MasonzeroDigitalWorks]

I don't know most of what I just heard, but I love hearing it. This stuff is awesome. Here I am with my NAS made from an old Dell Optiplex and feeling happy that I just bought a single 14TB hard drive. Maybe someday I will have a valid excuse to do any of this home networking stuff.

[@devinbridgwater]

This is literally how my Media Server started. ~10 year old gaming PC that was collecting dust, figured I'd get Unraid and play around with it. 4 years later I'm still using it, and have 32TB of storage, plex server, and other fun stuff.

[@TK-rs9lq]

Dead Guy + whiskey barrel chips? Time to figure out where to find some in Salem, then. Great tour!

[@beerdedvillain]

Love me some Dead 'n Dead.. It's amazing, especially if you hit one of their restaurants and get it right off the tap.

[@samjones4327]

Awesome Rack! Thanks for the tour!!! Cheers!!

[@SupremeRuleroftheWorld]

11:30 you know why rack rails are worth their weight in gold? because they cost about as much as if they were made of the stuff.

[@DigitEgal]

Could you compare the Cost of the Power Consumption aswell of yearly upgrades to servers aviable from hosters?

[@aniceto11]

Nice rack. But how much energy it takes and how much the energy would costs to keep a rack like these is US?

[@nphil93992]

Great video as always. Just one observation - everything needs a good clean

[@allan80supra]

You need to setup positive pressure filtered air in your garage to get those dust levels down! a few hvac hepa filters and two of the same blowers your using for heat extraction would do the trick, nice little weekend project and youtube video content!

[@javiej]

Great video. One question: Can you redirect the air conditioned exhaust to help to heat your house in the winter ?

[@CubicleNate]

Home Assistant is what I use. I can run it off of a Raspberry Pi computer. It is easy to set up and maintain. I also use Linkind switches that I flash with Tasmota to keep everything local. I totally understand the need of having local control over my local devices. Awesome rack, I tip my hat to you.

[@stephenreaves3205]

The UDM Pro SE is on the early access store and will have 8 POE++ ports and an internal SSD

[@Diversifried]

Out of curiosity, what kind of electric bill do you have with running all this hardware?

[@edbouhl3100]

Hope he’s buying renewable power for all that electronic activity! Otherwise….

[@thadrumr]

FYI Layer 3 Acceleration for CRS 3x devices is coming in version 7 of ROS which is currently in beta. Check out the release notes for version 7.1 beta 6. The current verison is 7.1 RC6

[@chrishooper7179]

I've been wanting to implement a PFsense router in my network. Those Zeus servers look tempting but I don't have a space where the noise wouldn't be an issue. Do you have any recommendations on something similar in price but quieter and able to handle gigabit lan? Figured better bang for my buck is used hardware rather than one of these all in one appliances going for $300 +

[@jlficken]

I just started a Threadripper Pro build in a CSE-836 to replace my current 2 x E5-2630L v2 CSE-846 UnRAID server. I have the case and motherboard now but am going to wait on the CPU and RAM until CES to see if the 3rd gen Threadripper Pro is released. If it's not I'll get a 3955WX. It should be a HUGE improvement. I also just grabbed a L1Techs KVM and it's awesome!

[@z4dude131]

I’ve just happened upon your channel after searching for home computer/server lab setups. Of course I have subscribed! I’ve worked in IT for the past 13 years and am finally setting up my own computing lab for fun and learning new IT skills/certifications. Looking forward to viewing more of your content. 😀

[@joran4612]

hey man, i live in the netherlands and we have this thing around christmas time called "surprise". its basically where you make a present for someone else of the group you do it with (chosen at random). i have my dad and im making a GIANT beer bottle and i wanted to ask you what beers you whould reccomend. my dad drinks a lot of special Belgian beers. i would like to get some weird beers and i have already chosen a few like a beer with a taste of a cake thing we eat here a lot, "caramel fudge stout", and "christmas ale". thanks already!!!

[@TheNerdy1]

If I am ever in OR, one stop would be to have a meetup and beer with Jeff! Love the content!

[@drumbum7465]

You mention that the AC unit only runs about 8 months out of the year. I'm curious what the coldest temps are in your garage during the remaining 3-4 months? I've been considering moving my computer equipment to the garage, but it can frequently drop down to below freezing (and occasionally below 0F) in the winter where I live, so I'm a little worried about that

[@joejoe6949]

I believe Lutron caseta dose use a hub for their proprietary clear connect but its its dependent on the internet.

[@flecom5309]

if you have an APC netshelter you should look at the ACF400, a replacement back door for netshelters with fans to force/exhaust hot air out attached ducting

[@DavidEsotica]

Great overview. I must admit that I'm a bit nervous that you don't have an off-site backup. But then ISP in USA are complete ass

[@ghomerhust]

so is most of the rest of our infrastructure. that's why so many of us build our own stuff!

[@Alphahydro]

Very informative and entertaining. Didn't realize you had that much processing power.

[@lmgendron]

There's one piece of equipment that you did not cover in this video... a big piece... and I would like to know about it, maybe it's obvious to everyone else, but I have no clue... What brand is your server rack cabinet? What are the specs? Except that it's a 42U.

[@lmgendron]

I guess it will stay a complete mystery for me.

[@VirtualDomain]

Big Fan of the channel, bit jealous of your setup ;) Got something similiar going on in my home, but what I'm struggling with is cable-managemet so I can actually "use" my sliding rails right... Maybe follow up video like a "how to cable manage in a rack"? Great Video, love the content!

[@Banner1986]

One question - what's your kWh pricing where you're at (electricity)? Have you done any research on potentially getting solar installed to offset the power draw by chance? We just bought a new house, which gives me the opportunity to fully re-think how I have my setup deployed, and power usage is one of those things that I've been considering doing something differently with. There's already 240v wired in as the previous owner had a home welding setup going... but I think maybe some solar panels might be in order here.

[@wxneck]

I see there's quite a bit of dust buildup on some of the lower devices. Do you have any processes or procedures for physical cleaning of your systems?

[@LrdLnitari]

This is amazing. What you could do is grab all the metrics from the entire rack from temps, cpu, memory, networks, whatever and be able to track it and monitor it on a single grafana dashboard. Probably can even host the grafana services on your systems. Again amazing. I do this for my current job.

[@TheJensss]

I'm really wondering why you don't upgrade to more professional router and firewall. I think pfSense would be much better than the ubiquiti router.

[@NotMuchHere]

Have you considered Vera / it is mostly self contained with limited internet connective requirement. Perhaps homeseer alternative (smart home controllers)

[@questionablecommands9423]

21:43 Guac-a-molly, the little known dip that makes you trip.

[@dillard715]

Unifi have a unvr and a unvr pro now for the cameras for more drive bays.

[@ianseiuli]

Where's the hot air exhaust exiting. I wanna see your outside part of your exhaust.

[@TransGalGaming]

anytips on getting a cheap UPS for rack mount to power a server, a PC and a TV/Monitor for graceful shutdowns?

[@EvanIIRC]

1. VA is not a battery capacity measurement, it's power. 2. Are you worried about dust at all? Seems to be building up pretty bad.

[@MatthewHill]

Why not run Guac in a VM or a Docker? Does it really need its own server--or is there some other benefit to doing it that way?

[@6LordMortus9]

I'm definitely interested in when you go to 240V!

[@peterchaploutskiy2555]

Good video. By pumping hot air of that room/garage, where is your inlet to compensate it? I guess hot&humid air from the outside?

[@Codeaholic1]

Question: when will the bottom servers get vacuumed out?

[@leadiususa7394]

As for your KVM solution, I used to have mine in the rack but over time I wanted to have side surfaces for my notes and other needs so I decided to move it to a full desktop surface 10 feet away from my dual racks. (refer to my home lab videos...) It was a great move for me. I can see my notes and so on and have room for space for anything I want vs. a tight format in my rack. BTW you could move this KVM mode back to your desk if you want. Just saying...

[@datashed]

I feel your pain on the Comcast front. I'm using a T1 line from GTT for upload and public-facing stuff because it was the only way I could get more than 5 static IP addresses. I have a /27 that's basically maxed out.

[@gary1143]

@craftcomputing the blue servers that you got for approximately 150 what brand are they?

[@makeitcloudy]

Some time ago you've been showing the K1 for the gaming rig, here I can see the gtx and you mention M60 GRID. Trying to understand whether this is proxmox based, with passthrough for the GTX, or you use the latest beta drivers for the GTX from nvidia ? can you please elaborate a bit on this ? That would be fantastic :D

[@gurgle9849]

Have you had a look at Shinobi for your self-hosted camera setup? I was thinking of going down that path, as I can still use the RTSP from Ubiquiti cameras.

[@alemaodownhill]

Cheers, I’ve been following since day one and what an upgrade you’ve made over time. Awesome lab. A good project for the future would be powering everrything with solar power. (Given that it is feasible in your region) That would massively drop your energy bill, and given that you are planning on installing a new apc it makes sense to future proof it.

[@CraftComputing]

I'm highly considering solar as part of an upgrade this year.

[@harmongoodman7432]

UDM-Pro SE will be coming out soon with 8 Gigabit POE ports with a few extra nice to have features including. Keep an eye out.

[@bartoszkazmierczak7249]

With the same 1 Gbps switch backplane as the original. Those ports will still be useless in many cases...

[@harmongoodman7432]

@@bartoszkazmierczak7249 Right, but it should work much better for an all in one NVR for home use. Esspecially if you're just connecting 1080p cameras to the 8 port POE switch.

[@nicolayleveille7401]

Do you use any 2.5 inch sas drives? If so, where is the best place to buy them?

[@neccros007]

Do you have a network diagram we can study to understand whats doing what better?

[@griffinjohnson4203]

That damn cloud neon cable is bugging me.... Can you put it through the rack mesh ?

[@elmestguzman3038]

I started experimenting with a PI security camera with PoE hat and an ELP camera with night vision. it may be an option at least for the cameras.

[@Hexers]

Out of curiosity... how much does the AC in the rack pull electricity-wise?

[@yakskiis7426]

I would love to know what your power usage looks like. Or anyone else that uses a rack. I have been thinging of buying one for all my equipment.

[@retarepo]

I would be curious about the electricity consumption/bill ;)

[@dawn1berlitz]

theres a reason why alot hate comcast it makes me glad im in a area not serviced by them and has the option for google fiber but i in reality dont have a backup for my server currently as i dont have the funds and also a place to store a server that backs up my media/backup server

[@tomwalshaw1881]

How much in electricity does that setup cost to run annually?

[@jothain]

So if I understood correctly you aren't hosting anything for others? It's hardware for personal usage only?

[@steveheist6426]

He was hosting a VM for his video editor / Rett, and two gaming VMs for his neighbor's kids.

[@jothain]

@@steveheist6426 Ok, that's pretty HC setup for such small usage. I was thinking that something like that could do something else, like whatever website hosting or something for some fee, but I did heard that there's not good up bandwidth, which obviously makes things harder. That's why I began to wonder that is it completely personal hobby server. Simply curious as I run proxmox currently on old Surface Pro 2 and one Celeron NUC. Just slight difference on hardware 😂

[@kyerie030]

Don't forget that your PBX provides the "cloud" LED system used to signify to your wife and kids that this is YOUR OWN CLOUD SERVER :D

[@dgr999]

I don't see the temperature display in the list, where did you get that?

[@TheGoatPC]

Curious, what are you running for the VM host on your storage and cloud gaming servers?

[@IPD2001]

Which kind of KVM Switch do you use? You didn't tell or did I miss it?

[@ewenchan1239]

Stupid questions: 1) Are there ways that you can consolidate and condense your networking stack that you've got going on here? 2) Can you show the back your rack and how the exhaust vent is connected to your rack in order to draw/pull the hot air out? 3) I assume that 17 degree F delta assumes idle or minimum loads, correct? i.e. if you were to run all of your systems at full load, that 7000 BTU of cooling capacity won't be enough to take all of that heat out (being that 7000 BTU is only 2.051 kWh of (thermal) energy). I am asking because I've got two systems that puts out somewhere around 3.4-3.6 kWh of heat, which means that my two systems alone is close to 1 ton of AC (12000 BTU or 3.517 kWh).

[@VioletGiraffe]

Is there some kind of integration between the UPS and the servers so that the servers handle power outage gracefully by finishing pending work and shutting down automatically? If not, I really don't see a point in having an UPS in a *home* server.

[@ArmBradPitt]

It's pretty critical for NAS systems whether its a synology, or qnap, or custom built, as pulling the power plug can potentially damage disks in raid setups. They each have the ability to perform graceful shutdowns when it detects power went to battery.

[@TechPewGuy]

I use a modem with a 2.5gbe port and the UDMP has a SFP+ module in WAN2 that will negotiate to 2.5gb while presenting to the UDMP as 10gb. Works well, when the UDMP is working well, that is. :/

[@videomasters2468]

do you have a dedicated electrical circuit to power all of this ?

[@GrimSpec]

Do you recycle the hot air in winter ?

[@experimental0000]

So much dust/dirt in that rack. How do you keep that from getting into the servers (assuming filters on the front of each unit, but want to make sure) and how often do you clean it out to prevent it from causing issues such as filters clogging up or settling on the internals if no filtering?

[@-ColorMehJewish-]

If you find a good home automation solution, please share. It's a nightmare trying to find products that easily integrated w/o being locked in a proprietary mode. Best I have come up w/ so far is to get devices that all use the same Zwave or Zigbee protocol

[@shykitten55]

I'm not getting why the ACU is at the bottom of the rack. Cold air sinks. Warm air rises. Ok, you have a fan and I'm guessing it is sucking the air out of the top of the rack, and that will cause the cold air to be sucked up. That's not good practice for air flow though - is it? Shouldn't you be blowing COLD air rather than HOT air? Cold air is more dense and so the fan works better. But I'm not an expert on the bigger picture.

[@richjfree]

might want to clean some of those intakes

[@chrisumali9841]

Thanks for the demo and walk thru, have a great day

[@willthelemon5681]

Yo I work in IT been at a couple notable companies in RTP in NC and just straight up understanding the jargon you are saying is awesome. Not to mention the amount of effort and tears goes into a full rack like that. A large kudos from me (not that you need it necessarily) but I wish I could manage to do something like that but HOA might have something to say.

[@ArthursHD]

I would like to put together an off grid system :) HD Plex PSU EPEVER Tracer 5kW model Charge controller Daly or some other BMS with custom parameters for cyle life. Prismatic LFP cells. (in place of ups) Collect all the production data to home automation system and adjust performance based on power available.

[@d00dEEE]

What about intake air filtering? I see lots of dirt collecting on the front of all the equipment, which ain't a good thing.

[@ME-fs8ir]

What KVM did you have that is not a full 1U ?

[@gregzavertnik1204]

I emailed your contact email but never heard back. Did you ever figure out how to enable hyper-threading on your Zeus servers. Your original video detailed that it was all supported but no bios settings to enable it or SR-IOV. Those issues prevent me from purchasing these servers and hope you can provide an update.

[@ZarMariano]

What do you do with the dust? any recomendations?

[@mikkelbreiler8916]

The cables were dusty, so he doesn't actually do much with the dust.