100 Gig Networking in your Home Lab

[@JeffGeerling]

8:31 time to upgrade your workstation, then!

[@RaidOwl]

Don’t do this to me, Jeff…

[@JeffGeerling]

@@RaidOwl Imagine, if you buy a modern Threadripper, you'd have enough PCIe for 400 Gbps...

[@RaidOwl]

@@JeffGeerling no...plz...

[@bluesquadron593]

@@RaidOwlYou know you must do it. You can always say the views will pay for it 😂

[@shephusted2714]

this is what everybody, even red shirt jeff, needs - overall a pretty good effort - pci lanes seems to be the limiting factor overall - the ws is the weak link. this video is a good one for biz - since they don't use gpu so much they could forego the switch and just do point to point from ws to dual nas - if you must have gpu in the equation then you need a server level ws/editing box with more lanes - what i got from this video - generally good content but raid owl does have to followup on this and optimize fully so he can enjoy more speed, save time, and be more productive, he may have to goto nvme arrays also

[@kumakell]

For the record, FS doesn't just "seem" like a good store, they are pretty much THE store for fiberoptic networking. They make EVERYTHING for fiber, and they're just about *the* go-to for third-party transceivers for the entire market. Their switches are used in enterprise as well, just at a much lower volume than their optics and accessories.

[@RaidOwl]

Cool! I’m not in touch with the enterprise market at all so this is good to know.

[@RealEricB]

I second this. As long as you are NOT TAA locked.

[@Bytional]

40G is a bit awkward right now, homelab users mostly not there yet but enterprise users are upgrading from it to 100G already.

[@nadtz]

People have been buying second hand 40gb enterprise switches for a while now. I first remember reading about that on the servethehome forums several years ago in regard to brocade and arista switches. It's 'obsolete' for the enterprise which just means you can snap them up cheap now second hand for homelabbers but you definitely need to do some research to not get a switch that sounds like a jet taking off and theres always power consumption to worry about.

[@rezenclowd3]

Enterprise is already at 400g...

[@TrTai]

They're nice if you can use them, but it gets a bit annoying because, yeah it's 40G but it's really 4x10G, as long as your tasks are multi threaded it can push it but I honestly struggle to push more than 2 saturated links and a bit of extra traffic on the other two. I'm sure some people can but it's really not as easy as you'd think. it does work nice for uplinks though still.

[@Darkk6969]

@@nadtz Careful on used enterprise gear as some of them need an active license to make use of the features.

[@kristopherleslie8343]

@@rezenclowd3even beyond that in labs

[@TrTai]

As a network engineer.... Just use the switch. The point to point to bridge is possible but it brings nothing but pain.

[@TheRealClutch1010]

I now have a way to describe my hobbies, crippling inability to be content with what I have.

[@CraigMcIntosh]

@TheRealClutch1010 I know what you mean, just starting to build a home lab and home nas

[@truthfromearth]

Basically.

[@TDCIYB77]

iSCSI should be the best option to improve speed for editing. And supported by both TrueNAS and Windows.

[@pietpaaltjes7419]

That would be interesting.

[@BattousaiHBr]

iSCSI would be the most performant for him, but the downside is fixed space allocation for the share since it's not thin provisioned. alternatively he could just migrate from SMB to NFS and see a nice performance bump with very similar configuration.

[@torgrimt]

looking forward to your review of the threadripper in a week ;)

[@russgifford]

Just came here to say that ...

[@BillLambert]

Threadripper owner here: you still need RDMA to reach any meaningful file transfer speeds. Network packets are typically 1492 to 1500 bytes, or 9000-ish if using jumbo frames. That's between 10 and 66 million packets per second for 100GBE, which is a massive amount of overhead for the CPU. RDMA instead takes megabyte or even gigabyte-sized chunks of data and "packetizes" it right on the NIC instead of your CPU, and it does so at line speed with dedicated silicon. Well, assuming your RAM and PCIe bus can keep up.

[@cjchico]

I use this switch for vSAN, breaking out to 2x 25GbE per each Dell server. Works great and haven't had any issues yet. RouterOS is very odd and definitely takes some getting used to.

[@phillipmartinez583]

When's the threadripper video coming out?!

[@RaidOwl]

When AMD sponsors me

[@juniornunes]

the "see i fixed the saggy servers!!!" gesture was great lol

[@dagamore]

by adding in a 2x4 or something like that ..... like I did at my home lab it worked.

[@juniornunes]

@@dagamore hey ive done a shelf and when th shelf didnt work i used zip ties it aint stupid if it works only if it fails

[@ajhieb]

I had a similar upgrade path, where I started with 3 servers connected via 3 dual port ConnectX-3 cards, but wanted to connect to some other stuff, so I found a Mellanox SX-6036 40Gbe switch on eBay for about $150 and I've been thrilled with that thing. It's not "sit next to it" quiet, but it's not even close to being the loudest thing in my rack. If you're not averse to used equipment, it's a great option.

[@CassegrainSweden]

This must fall under the category: do I need it, hell no, do I want it, hell yeah 😂

[@RaidOwl]

That’s 99% of my life

[@DeNNiiiable]

Yup. How I end up with most my stuff

[@ajpenninga]

This is @RaidOwl's YT channel slogan

[@MM-vl8ic]

I've been using the ConnectX-3 40/56Gbs for 5 plus years with a Mellanox SX6036G ("G" = Ethernet)..... I found when testing, it's best to use 100G+ ram disc on each machine to eliminate storage device issues..... also being a bit retro... Asus X99 workstation MBs with Xeon e5-1660/1680 v3 over clocked 4.2Ghz with 2666 ECC ram 40 PCIe lanes or more with the PLX switch chip, other MBs are Supermicro X10 series... Remember those Enterprise cars need air flow.... I use retro desktop cases with a side fan to cool NICs, Raid/HBA cards and NVMe drives....

[@AndrewWells527]

9:30 I'm disappointed that the next cut after "that would be stupid, right?" wasn't with those PC parts on the table. Current tech really isn't designed for single transfer speeds at that level. That switch was probably designed to be at the core layer of a network. Still fun to play with though.

[@RaidOwl]

Yeah if I really wanted to build out an expansive RDMA-enabled network that would require a different switch…or a bunch more NICs lol

[@iroesstrongarm]

That "oh god" from you wife definitely had undertones of her full awareness that too much money was spent. 😂

[@chaosfenix]

I love this video. Yes I know there isn't a ton of use for it now but I think there really could be if it was more common. I would love to see some sort of direct video output over the network solution. HDMI 2.1 is only 48Gbps and Displayport maxes out at 80Gbps. Currently remote video is handled by encoding the video signal with something like h.264, sending it on the network, and then decoding the video before presenting the picture on your display. I would love for there to be an option where we could skip those encoding/decoding steps as those add extra latency and can introduce compression artifacts, and instead just send the entire video stream on the network with some overhead left over for USB peripherals. Again, I know this is a stupid upgrade now, but I would love to see this tech better utilized in the future. We have been stuck on 1Gbps for so long that it limitation itself has impacted what we can do with the bandwidth.

[@inkprod]

There kinda is already! In professional broadcasting they use the SMPTE 2110 standard, which is essentially raw video (SDI) over IP. That stuff doesn't exactly come cheap though and you need things like PTP aware switches to support it.

[@chaosfenix]

@@inkprod That is awesome. I tried finding something but all I was finding was boxes that looked like they still encode and decode the signal. This looks like exactly what I want. I wonder how open the standard is. Like are we talking open like AV1, restricted but not bad like h.264, or god awful like h.265? You don't actually need that much bandwidth either. I said 80Gbps for Displayport but it really depends on your resolution and refresh rate. A 4k120 direct stream is only 26Gbps and a 4k240 is 55Gbps. Sure there is going to be some overhead for encapsulation but you still have enough for a full 40Gbps thunderbolt 4/USB4 connection right along side it. If you actually had this networked through your home you could have a single virtualization server in your network closet and simply have cheap endpoints everywhere else. Even if you were remote it wouldn't be worthless. I am a lucky one and have 10Gbps symmetrical home internet. I could use either a 1080p144 or a 1440p85 video stream with that. Sure, you are depending on your remote site also having that kind of a connection but a 1080p60 stream is only 3.2Gbps. Give it a few years and that may actually be doable.

[@kjeldschouten-lebbing6260]

Its a super nice switch, as you can have: - Redudant 100g uplinks - 4x25g for servers and workstations - 4x10g for legacy equipment or 1g switches with 10g uplinks

[@RandomTechWZ]

10gb networking seems to be the sweet spot for home labs and your wife sounds exactly like mine when I tell her changes I made to my home lab lol.

[@RaidOwl]

100%

[@jeremybarber2837]

Good stuff right here. Greatly appreciate the ride along as you learn.

[@Prime0pt]

What is really interesting will this mikrotik work under full load? We had few mikrotik switches and had to wait two years before mikrotik made SwOS for them. On routeros they just weren't work under the load.

[@be-kind00]

I agree!

[@Jinix64]

If you make a Filesystem that host the SMB run in Async mode on the Truenas it will go way faster. However, do note that this is not advised for sensitive data, as you can lose data on power failure. But it will fly network benchmark wise.

[@gigabit9823]

By far the funniest Homelab channel on YT. Never change my Texan neighbor.

[@adrian32772]

we know it's a PCI bandwidth issue, but make sure you are also using iperf3 with WSL on windows since the windows compiled versions are still using a translation layer for how it's complied. @technotim just went over this in his latest Unifi video too.

[@byronservies4043]

The only reason I am upgrading to 10gb at home is that I am finally terminating the OM1 fiber I installed during a remodel. That was in the year 2000. Also re-terminating the cat 5e while I'm at it. Lots of 100g at work, though.

[@Twitch_Blade]

would be a cool setup for prox mox cluster

[@BrunodeSouzaLino]

For those interested. To convert from bps to B/s, you divide the value by 8 (100 Gbps gives you a theoretical max speed of 12.5 GB/s). To convert from B/s to bps, you multiply by 8. 8:16 That would be 8GB/s, which would be ideal to run 4 10GbE connections, but you'd need at least x8 for 100GbE. Though you could also go for a PCIe 2.0 x16 slot, which has the same speed.

[@RaidOwl]

At 8:16 I mentioned it’s x4 of gen3 speeds since the card is gen3. But yes x8 of pcie4 would suffice.

[@BrunodeSouzaLino]

@@RaidOwl 8 GB/s is the max transfer speed for PCIe 3.0 x4.

[@tommybronze3451]

If you want better speed than samba, than maybe try NSF ? another option is iSCSI but I'm not convinced it would outperform NFS ... but hey, try and see.

[@Rostol]

the "problem" is the QSFP modules you are using. there are 2 kinds of 40gb modules. SR4 and BD (Bidi)... the 1st kind uses the 12 fiber connection 100gb fiber uses (MTP for multi pair). the BD one uses "nornal" 10g-20g fiber (2 pairs, normally LC terminated) that kind has 2 20gb channels. so the maximum per connection is 20gb or rather it can work at 20gb for 2 simulatenous connections totalling 40g. those 100gb cards need server airflow, install a noctua on them. they wll die. i killed my 1st 40gb card like that.

[@drubizzy]

I have to wonder if some of those GPUs we've been seeing with NVMe slots are a preview to seeing more types of IO bundled into the GPU package to take advantage of the over-allocation of PCIe lanes to the top x16 slot.

[@thesussypupper]

Switches are actually a good idea. it allows more granular control. layer 2 or layer 3. Each installation has different requierments. not to mention any changes without a switch brings..... longer change windows, roll backs and other problems as well. not to mention the right switch can do routing to offload the work from a main firewall or router. better speed and versatility in the end. Switches allow better scaling and future growth with minimal cost and planning as well. in a mission critical network like 911 for example; you need complex enough to be secure and work at scale, but simple enough to maintain and have "eyes" on the complete end to end system.

[@LakeZern]

On my Asus S14NA-U12 / Epyc 8024P a transfer to a PCIe 5.0 M.2 NVMe (T700) archives almost 5GByte/s now. Looks like the CRS504-4XQ-IN is much better than we have thought? Maybe we all have underestimated the "limited" write / read speed of the drives used?

[@lpgsk]

The power supplies seem hot swappable, are they? Cheap networking gear is notorious for not having hot swappable power.

[@mikealthomas1]

I wish Mikrotik made a 8-Port * 100 Gig switch. But overall, a really good 4-Port switch

[@Darkk6969]

They will eventually. I have several of their switches and they're great!!

[@mrkdosmil2879]

Not sure if it's just on my end but it seems like you need some sound dampeners. I can hear a bit of room reverb in your audio.

[@LakeZern]

Any news on Epyc mainboards? There are interesting products like the Gigabyte ME33-AR0. AMD 8004 series comes with 96PCIe lanes...

[@NightHawkATL]

I am still trying to realize 10G on my servers and you're over here with 100G lol. Great video and break-down of what was involved.

[@msvaughan]

100G network is one of those things that I feel is overkill for a homelab. 40G is ideal considering that you probably won't have media that goes that fast even over SMB connections. Personally, I have a 1G connection that is more than adequate for what I have. Thought about 10G which I have in the future but I am not using the bandwidth I have at the moment.

[@cameronfrye5514]

Well, considering you found a path through RDMA to achieve full network speeds in your Windows workstation, when does your new motherboard ship?

[@RaidOwl]

Tomorrow

[@LakeZern]

Have you checked NICs (e.g. Intel) supporting iWarp? Mellanox cards do not, but Microsoft seems to support iWarp. My understanding is that iWarp does not request any support by the switch

[@rdsii64]

Yes, its worth an all flash server just to get that good RDMA throughput.

[@SB-qm5wg]

Mikrotik has been good to me with the value to price

[@PeaceIndustrialComplex]

I've been wanting to get my hands on one of these switches just because 100Gbit sounds so insane

[@perjernstrom5178]

How about RDMA GPU? Sort of next level for “cloud” (read home lab) gaming.

[@acenio654]

Recently a listing popped up on my country's main used market website for a 16 port qsfp28 switch for just under $700. Tempting

[@galvesribeiro]

I’m curious how did you setup physically the RDMA mode. I mean, if you are passing thru that switch, it would have failed. AFAIK, the RDMA support requires all parts involved in the path like network cards, switches, routers etc to be RDMA compatible and properly configured. The Mellanox cards and the switch you are using are the same as mine and Mellanox use RoCE for RDMA which requires a bunch of features from the switch that Mikrotik doesnt have implemented on any of their products. Would love to see more details on that RDMA setup. Thanks!

[@RaidOwl]

It was direct from each machine. Didn’t go through the switch

[@seethruhead7119]

Been coveting this switch for a while. Haven't pulled the trigger but still thinking about it.

[@RaidOwl]

Do it ;)

[@geesharp6637]

What about NFS instead of SMB? I've seen some videos recently about NFS on Windows performance is great.

[@GregBrantUK]

"If you're still watching you're Infiniband"... I can't tell if that's a complement or not

[@RaidOwl]

😉

[@mosquito7450]

run the gpu on fewer pci lanes. Linus tested this. No performance hit on 4 lanes.

[@krisclem8290]

"That would be stupid, right" seems like forshadowing to me.

[@TheComputerNerd248]

Are there 2 of the new UDM Pros with the 2 HDD slots in your rack?

[@RaidOwl]

Yessir

[@marcinkrasowski7841]

Z790 supports pcie bifurcation in 8/8. So you split your main x16 PCIE 5.0 into two slots: one for the GPU and one for the NIC. You would need a passive adapter to go from single x16 to dual x8 physical connections but you wouldn't be bottlenecked to 32Gbps without buying a new system

[@RaidOwl]

Not all z790 boards. For example, the one I have…lol

[@kristopherleslie8343]

Good video buddy cheers on the next beer

[@alexwhitehouse1958]

With your crystalkdisk benchmark showing as exactly 10gb read and write that seems a little too coincidental to me. You’d expect reads to be faster than writes. Are you sure there isn’t a bottleneck or something like a port still in a 10gb mode?

[@RaidOwl]

Nah I only had the single interface enabled which is the same one I did the iperf tests with. Could be that my pool isn’t properly optimized

[@bradleystannard7875]

Mikrotik just go "how many gigs can we ram in under $700" and then this was born

[@Girgoo]

Would NFS help with the performance issue?

[@RaidOwl]

Don’t think so

[@guy_autordie]

I'm MAD and angry at the PCIE lane limitation. Before, we could have all the PCI goodness we wanted. Now if you don't get a server grade MB&Proc, forget your cheap homelab form your older Gaming stations in which you could add an add-in card for that stuff you wanted to do.

[@Darkk6969]

One of the reason why I am keeping an eye on a decent deal for used AMD Eypc server CPU paired with used SuperMicro motherboard and ram. They're getting to be cheap on e-bay.

[@computerenthusiast402]

What network cards and switch do you recommend for a PC network ?

[@ryanamberger]

Should be able to remove ether1 from your default bridge and still use it to power the switch with poe. Just remove from default bridge and get a 48v/48w Ubiquiti POE Adapter and use that POE output. Should work. May verify the adapter is (4,5)+/(7,8)- to be safe. Some of ubiquitis use 4 pair. May be fine, Mikrotiks datasheet should show exactly what poe it wants. That is, 2 pair or 4 pair and the watts required. I'm too lazy to look. Just POE port on adapter to POE in on CRS. Leave adapter LAN unpopulated.

[@rrsf4i]

use a pcie bifurcation x8/x8 since your gpu doesnt use those 16lanes (trust me, 1060 had 3 less frames on 4 lanes vs 16 on pcie gen 3, google your card and see how many lanes it actually needs). i do the same, invent reasons or hide behind semi legitimate ones but the truth is I WANT A FAST SW vs i need it

[@RaidOwl]

Yeah looked into this but mb doesn’t support it

[@rrsf4i]

pls dont take it the wrong way, i have a soho/home lab and i have 3 mikrotiks with fibre runs between rooms or DACs and a 4th one ordered. 😂

[@DeNNiiiable]

​@@RaidOwlcan't you plug GPU in other slot? As even 4x gen4 is prob ok or 8x gen3

[@kumakell]

@@RaidOwlI'd be kinda surprised if it really doesn't. Usually it's automatic, without an option, if there are multiple greater-than-x4 slots on the board. If you plug ANYTHING into the second, it downgrades the first to x8. What's the model?

can you not put switch os on them?

[@christianponopp8756]

Do you use the highest SMB Version in Windows and deactivate SMB 1.0 in Windows? Windows might fall back to SMB 1.0 if it's still active

[@JTM_djg]

Bar Harbor?! Visit or live about? Central Maine here!

[@RaidOwl]

My mommy got it for me when she visited

[@pewpewpew8390]

its sfp28, not sfp+ ;)

[@DefconUnicorn]

It doesnt have to run routerOS you can reboot it into SwitchOS mode. Also try MTU 9000 and research jumbo frames.

[@oscarcharliezulu]

Addiction to tech is not a joke people. Brett needs our compassion, not our derision.

[@polandlp5965]

Turn on Jumbo frames for better speeds

[@LiLBitsDK]

love videos like this but atm just on 1gig... planning to swap to 2.5gig "soon"TM

[@RaidOwl]

2.5G will be nice 👍🏼

[@DeNNiiiable]

10gb second hand gear is real cheap SFP+ and ethernet end up similar but 10baseT enternet is more flexible. switch is more but cables cost less

[@PrimalNaCl]

I have 2 of those Mikrotik switch and an Arista 48 100G qsfp28 ports and several intel e810-cqda2 cards. Genreral desktop machines will be pcie lane constrained. I can eek out about 50G. You need something that has the full 16 lanes of gen4 pcie to get the "love". Windows has a larger perf hit than linux and wsl2 blows all the goats. 100Gbit is a beautiful thing!

[@studioxxswe]

well I work in the broadcast industry, SMB 3.0 and beyond is for sure not limited to 10G by any means. Might be Samba or whatever TrueNAS is using that causes that, not the protocol

[@FunkyKong]

Yeah, SMB is definitely capable. You should not be seeing a nearly 1/10th performance hit. There is something else wrong. RDMA is only going to work point-to-point in your setup since the Mikrotik has no support for it.

[@DeNNiiiable]

NFS shares faster? SMB is slow but I think your bottleneck might be the array but not sure. Maybe setup a nvme single disk and repeat test. Regarding the lanes your graphic card is unlikely to be much slower on 8 lanes. It's the enough ram for Arc available and is the nas CPU the bottleneck?

[@computersales]

I'm disappointed you didn't cut to the new threadripper build after talking about it. 😂 Also yes all flash NAS is totally worth it. As long as you don't do it the dumb way mine is set up for now. 😅

[@bendertube254]

I think, to have proper rdma through RoCE you should have a RoCE compatible switch. Probably with Mikrotik you should go with iWarp, but not everyone network card support iWarp.

[@RaidOwl]

Right. I just did a direct connection for testing.

[@AfroJewelz]

at same time , it' seems most rdma roce NIC is only proprietary OS compatible and lack of open source fabric driver support. those proprietary shit are expensive like hell

[@guyfeldman4697]

It’s not time to get one new system, it’s time to get 2 more hl15s to build a ceph cluster and test against an nvme rbd pool

[@jannikmeissner]

I went way crazier when I got two Mellanox / Nvidia SN2410 switches… BUT I use GPU Direct at least.

[@RaidOwl]

Boy you WILD

[@jannikmeissner]

@@RaidOwl Can't wait when the new 800G stuff gets tossed out at work… 48 months, I am already counting!

[@pewpewpew8390]

checkout the multithreaded iperf3

[@SmalltimR]

Can't wait to see your Threadripper update :p

[@JPEaglesandKatz]

LOL... Wish I had the money to spend on a rack or on the stuff you are getting... To just blow it on stuff I don't need.... Nice video... :) Had a good laugh as well (all positive tho)

[@Th3K1ngK00p4]

I get SFPs from FS regularly, definitely a reputable store.

[@DarkNightSonata]

awesome video. I really like how chasing performance make us learn new things. one question, while the switch is 100gb, how does the router impact the performance of the network, aren't packets sent to the router and back or are they communicated directly between each other from the switch itself ? if directly, then how are packets filtered or firewall rules apply ? thanks.

[@RaidOwl]

As long as they’re on the same VLAN then the switch will route by MAC address so it won’t need to go back to the router.

[@CyberSquatch007]

I'm surprised how many people use their networks for just one purpose... I would absolutely use 100Gbps speeds with all the different things running in my network stack.

[@ws_stelzi79]

So next there will be a new Threadripper build to get enough PCI-E lanes to have a "proper" 100G networking to your NAS for editing videos! 😉😏

[@Georgio_TheChef]

I just set up my 1 billion gig network, it's so dope. I can dowload the universe in 4 minutes

[@RaidOwl]

That’s a lot of recipes Georgio

[@canonwright8397]

"Hey babe, one hundred Gib a second." 😎 Slap! "Ok? Maybe I should have said a thousand Gib???" 😏 Keep up the good work, Raid.

[@theiaminu5375]

Cool , when MS borks your system with an automatic update , it will occur that much faster !!!

[@phucnguyen0110]

Love the content, Brett! I guess I have to give Colten aka Hardware Haven a shoutout for me discovering your channel and learning about home lab/home server in general :D

[@fwiler]

Is it worth it? Do you really need to ask the question? It's always worth it. I would be curious about power consumption difference between 10,25,40,100 including the nic and the switch.

[@RealEricB]

You should have used a 100g to 2x40g dac to save ports. You do you though. Just know, that's an option for future expansion.

[@matthew-d3p]

looks like an intel problem not a pcie problem 3:17

[@moe85moe85]

Time for a new workstation and an all flash windows server so!

[@greenprotag]

Can you get full RDMA access with a ZFS set up? I guess if so you are limited by client speed? Could you use a ram disk on client side? Or reading an NVMe raid zero array? ...hmmm RDMA wants windows on both ends? ok.... hmm. So is the transfer direct from memory to memory? so as long as your memory pool is large enough?

[@kwith]

Recently upgraded to 10GB at home and now my bottle neck is the damn hard drives! hahaha

[@AngryGibberish]

Ok, SMB Direct is cool. I had the thoughts of: a) It would be cool if Unraid supported this. to b) There's probably no way for Unraid to support this lol

[@tyler5888]

Nothing is worth the hassle of running Windows Server in a Home lab. You'll save more time with the slower network speed vs troubleshooting and Windows updates with Server 2022.

[@arcanocito]

GPU in the 4x Slot it is

[@hoboactual]

Can someone explain to me why these are so cheap?

[@icebalm]

The biggest issue with 10+GbE is that without RDMA the CPU has to process packets, and unfortunately these Mikrotik switches don't support DCE or RoCE at all... With that you would get vastly superior performance with SMB Direct.

[@RaidOwl]

Correct, but that FS switch does 😉

[@inmy30s]

it keeps escalating .. you need more sponsors 😂

[@ILike2Reed2]

I think it's absolutely worth it for YOU to make an all flash server to saturate that juicy througput for no other reason than that we can watch and live vicariously through you

[@CeleronS1]

Although it's very cool. I would say that it is extremely wasteful. I would rather install large capacity nvme drives on all clients and then allow some kind of caped speed (like half a gig) sync between all of them. I think it is just better way to do it. You got instant access with no network bottleneck on your local drive while any updates you make gets synced slowly over time. Centralised stuff is cool, but you will always get congestion. I think If spend like $1.2k on ssd between all systems, I would become victorious over user experience.