Windows vs 30 Hard Drives - Can it Do it!? Find out!

  Рет қаралды 194,024

Dave's Garage

Dave's Garage

Күн бұрын

Пікірлер: 891
@montagistreel
@montagistreel 2 жыл бұрын
Dave, the closest thing we have to a tech Bob Ross
@jeffhex
@jeffhex 2 жыл бұрын
with better hair.
@marvintpandroid2213
@marvintpandroid2213 2 жыл бұрын
Happy little cloud (storage)
@gordonm2821
@gordonm2821 2 жыл бұрын
Bob Ross but using Windows Paint
@BTheBear
@BTheBear 2 жыл бұрын
We don't have data loss, we just have happy little accidents!
@michaelhanson5773
@michaelhanson5773 2 жыл бұрын
Happy little bits on happy little platters, written by happy little magnets.
@philarmishaw3730
@philarmishaw3730 2 жыл бұрын
The Garagenator
@Gabber1110
@Gabber1110 2 жыл бұрын
The Garage-inator ?
@zilog1
@zilog1 2 жыл бұрын
Grarageenatorinator
@craftsman123456
@craftsman123456 2 жыл бұрын
I just love seeing how you don't give up on getting it running under Windows
@alisharifian535
@alisharifian535 2 жыл бұрын
Because Microsoft recommends users(developers included) shouldn't bite the hand that used to feed them.😁
@AdrianNelson1507
@AdrianNelson1507 2 жыл бұрын
@@alisharifian535 🤭 storage isn't my thing but I was sat here shaking my head. This limitation wouldn't exist in Linux! But we've all had a crazy summer with Windows that we're not proud of
@MrBlaDiBla68
@MrBlaDiBla68 2 жыл бұрын
Yeah, there were really some "Take the pain!" moments ;-))
@debbiebernhardt3742
@debbiebernhardt3742 2 жыл бұрын
There is 26 letters, so the limit is 26 based storage.
@palomarjack4395
@palomarjack4395 2 жыл бұрын
... even though it is a an obnoxious amount of work. I slapped together a Linux server under the Mint distro, of all things. Brought the storage drives into the directory tree in FSTAB, changed the permissions to share them and there is my network storage. Took about 5 minutes. The best part, I don't have to be concerned with Windows idiocy, security issues and cost. "It just works."
@zippymctarget2770
@zippymctarget2770 2 жыл бұрын
I had three of the 45 drive units at work a few years back, it was a good learning experience for configuring ZFS for data throughput.
@DavesGarage
@DavesGarage 2 жыл бұрын
Can you summarize 3 years in a comment for me? Just the good stuff 🙂
@sszerotosixty
@sszerotosixty Жыл бұрын
By default the cluster size is set to small (4kb). If you reformat the pool with a larger cluster size, say 256kb or 512kb or even higher you can go to 256tb volume size with NTFS. And your performance will be MUCH improved.
@andrewfidel2220
@andrewfidel2220 Жыл бұрын
Nope, the problem was he was using win10 instead of Windows server, storage spaces in the consumer/workstation builds is artificially limited to 63TB to force you to buy a server license.
@THEhairfarmer
@THEhairfarmer Жыл бұрын
I suspected a licensing limitations - have been there several times before
@ericbsmith42
@ericbsmith42 3 ай бұрын
​@@andrewfidel2220 I'm currently running Windows 10, and my current Storage Spaces has 92TB of drives in it. I Thin Provisioned my main drive to a 100TB Partition using dual parity on the Storage Space. In order to create larger volumes, dual parity volumes, or to change the Storage Space Interleave size (necessary to improve performance) you need to create your Storage Space using PowerShell commands.
@ericbsmith42
@ericbsmith42 3 ай бұрын
Manually setting the Storage Pool Interleave size and then formatting the partition to use a Cluster Size that aligns with the Interleave value will drastically improve performance. For instance, I have a 10-column Dual Parity Storage Space. That means it has 8 data columns, and I set the Storage Space to use the smallest possible Interleave of 16kb. 16kb * 8 = 128kb stripe across the 8 drives, so I formatted NTFS with a 128kb Cluster Size. 1 NTFS Cluster will write 1 stripe across all 8 data strips. The reason why 256kb or 512kb clusters work better on a default Storage Space uses 256kb Interleave, so those extra large clusters come much closer to filling an entire stripe across all the data columns, which drastically improves parity calculation. But actually aligning Interleave size to Cluster size improves it even more, however in order to set those parameters in Storage Spaces you need to create your Space using PowerShell commands.
@jjones2582
@jjones2582 2 ай бұрын
@@andrewfidel2220 - I wonder if the Workstation SKU that includes ReRFS would have the same limit without having to move to Windows Server.
@NeverlandSystemZor
@NeverlandSystemZor 2 жыл бұрын
As a big fan of LTT... that gag was soooo great!!! The ONLY thing missing was dropping something.
@darylsmart9459
@darylsmart9459 2 жыл бұрын
If you want to remove partitions with diskpart, one you have selected the disk, type "clean" and it will wipe all partitions from the drive.
@marcellipovsky8222
@marcellipovsky8222 2 жыл бұрын
I was wondering the same, why he did not use it. That would be much faster.
@Tularis
@Tularis 2 жыл бұрын
Ikr
@psjoshooaj
@psjoshooaj 3 ай бұрын
I was thinking the same thing but sometimes for a one-off experiment it’s just not worth the time to look up how to do it when that might take longer than a bit of click-ops.
@dustojnikhummer
@dustojnikhummer 2 жыл бұрын
I'm here building a 8TB (12TB raw) NAS and I'm very happy. Seeing big hardware like this makes me jelaous. And then not, realizing how much it would cost in power to run.
@mariusvoicu683
@mariusvoicu683 2 жыл бұрын
About 20% capacity out of a 2000w ups
@jovetj
@jovetj 2 жыл бұрын
*envious
@ThePlumbeus
@ThePlumbeus 2 жыл бұрын
You might want to look into the PowerShell commands. It not only saves time, but also allows you to set custom settings for your storage spaces. The GUI doesn’t allow you to set many settings like number of parity drives, columns/stripes and tiering (available per GUI on Server) which allows you to set the equivalent of other raid modes. The GUI is ok for normal projects, but PowerShell is really needed for these kinds of systems.
@mrmotofy
@mrmotofy 2 жыл бұрын
So what was the 64TB limit he hit?
@RonaldSchneiderGiebenach
@RonaldSchneiderGiebenach 2 жыл бұрын
I only once tried out storage spaces. Couldnt get warm with the UI that tells you nothing what happens, I dont know how I would be informed that a drive failed. And if a drive failed, I wouldn't know which of the drifes is failed. It should really have a guided UI (mark disk as 1 and plug it in, and restart system, then disk 1 is inserted. Do you want to add another disk? That way in the event of the failure there would be the possibility that you know which drive was failing) In my test one drive failed and windows was trying to resync it for months qithout telling me. I am lucky I catched it by randomly opening the storage spaces UI...
@attilavidacs24
@attilavidacs24 2 жыл бұрын
In Disk Part you type in select disk (disk number) and then type "clean" to remove all the partitions. You can create a batch script and do all your HDDs in no time.
@adamg8588
@adamg8588 2 жыл бұрын
Also the LTT comment, youre hilarious. Well done Dave. Really enjoy your videos man. Thanks for sharing. Always a thumbs up for you
@Holek2
@Holek2 2 жыл бұрын
Hilarious, like this segway... to our sponsor!
@judenihal
@judenihal Жыл бұрын
i hate ltt
@Maty2001
@Maty2001 Жыл бұрын
@@Holek2 The sponsor: "Windows NT is the new OS, built with my own hands, here at Microsoft!"
@user-cx4ev9fw1k
@user-cx4ev9fw1k 3 ай бұрын
agreed
@alanmusicman3385
@alanmusicman3385 2 жыл бұрын
I remember when I was at DEC and first got hold of an RZ26 (3 inch drive, came out about 1995 or so) and I held 1GB of data storage in my hand and (because I went all the way back to 10MB cartridge hard drive RK05s and their equivalents) I was completely awestruck! Now I have 64TB on my video server and I barely think about it - except that I need to upgrade it. No wonder the world has changed so much eh?
@n6cid
@n6cid 2 жыл бұрын
Remember back in the day when ordering new drives thinking to yourself "I'll never run out of storage with this drive"....:)
@DavesGarage
@DavesGarage 2 жыл бұрын
Ironically it's mounted less than 6 feet away from my Commodore D9090 10-megabyte unit!
@JohnnieWalkerGreen
@JohnnieWalkerGreen 2 жыл бұрын
Mine was a DEC PDP-11/84 under RSX-11 with two 10MB RL-02 disks and a DECUS C compiler. No virtual memory: that was when the programmers did the memory overlay design and calculation.
@sbrazenor2
@sbrazenor2 2 жыл бұрын
I think my first HDD was a whopping 40MB.
@n6cid
@n6cid 2 жыл бұрын
@@sbrazenor2 When I was stationed in Germany my first dos box was a Commodore Colt with a 20 meg HD. When we got deployed for operation DS/DS I ordered from the States a 540meg HD (ERLL) with adapter card. That was the beginning of the long journey to include Linux, Free BSD, Red Hat and the list goes on and on...:)
@larryblount3358
@larryblount3358 2 жыл бұрын
Granted a 14/15 wide raid 5 is nice. I think for a server based solution a raid 6 based raid is better. Allows for two disk failures. Small loss in overall user disk space for a host of other failure and failure performance benefits. Enjoy your videos. The technical depth of knowledge is great. Thank you.
@DavesGarage
@DavesGarage 2 жыл бұрын
I'm thinking when I add the other drives I will split it into 8-wide ZFS2, but am not 100% sure yet!
@SlevinKalevera
@SlevinKalevera 2 жыл бұрын
Hearing that you and Jeff are communicating makes me so happy. And I can't imagine how much he's geeking out talking to you. LEGENDS!!!
@mikkelbreiler8916
@mikkelbreiler8916 2 ай бұрын
Aaaaand no we want a collab.
@RichardDzien
@RichardDzien 2 жыл бұрын
As someone who uses Netapp storage and the like at work, it's interesting to see how good the open source solutions have gotten over the years.
@ambostralian
@ambostralian Жыл бұрын
Describing something like a striped array as "dangerous" instead of something like "most risky" might just be my new favourite thing
@Flash2171
@Flash2171 2 жыл бұрын
Maybe I'm being paranoid, but I'd be concerned about the vibration from the compressor running, causing issues with the spinning rust drives. None the less, this is an awesome project and I enjoy your channel a lot. Congratulations breaking 285K subscribers!!!
@andrewfidel2220
@andrewfidel2220 Жыл бұрын
Perfectly valid concern considering you can cause a temporary lockup in large arrays by yelling at them.
@josephking6515
@josephking6515 6 ай бұрын
What about the dirty power with the compressor constantly cycling? That would concern me.
@danielfisher1515
@danielfisher1515 2 жыл бұрын
Every compressor I've been around is not only very loud, but very percussive noise. I'd be worried about the effect on long term disk life.
@Tofflus
@Tofflus 2 жыл бұрын
I'm not the only one who thinks that could be a big problem :O
@stevedixon921
@stevedixon921 2 жыл бұрын
Valid concern. Heard stories of people 'carting' a stack of drives from one building to another and the vibration from the ground through the cart to the drives was enough to cause damage to the drives (makes sense if you think about it after the fact). For this installation adding some anti-vibration to the mount would be advised, though it being 'detached' from the wall and floor will help. Alternatively there may be anti-vibration mount options for the compressor. Time will tell, but if the drives fault randomly (or fail) it is something to consider. I know two customers who have their servers next to water sources, so, sometimes you make do with what you have.
@PaulMawdsley68
@PaulMawdsley68 2 жыл бұрын
Hi Dave, thanks for creating this channel and sharing your experience and wisdom. In this video you mentioned, when deleting the partitions, "and you can't use diskpart". I was a little stumped by that. Why not? Using Truenas for my setup, I can't test this, but I would have assumed you could select disk xx; clean.
@natemauger9757
@natemauger9757 2 жыл бұрын
Right? I was thinking EXACTLY the same thing there!
@eDoc2020
@eDoc2020 2 жыл бұрын
I would need to assume he meant that you can't do bulk operations in diskpart.
@valuedhumanoid6574
@valuedhumanoid6574 2 жыл бұрын
Mine is 500 gigs, so I think you've won this contest. Holy smokes! I didn't know they made them that large.
@jfniv
@jfniv 2 жыл бұрын
I spent last weekend digging online and reading about maybe setting up a virtualized truenas trying to get a feel for ability, options, possibilities. This video is an excellent addition to my research, thanks again for the content you produce; it's much appreciated.
@wishusknight3009
@wishusknight3009 2 жыл бұрын
I have ran truenas/freenas for many years on bare metal. I also have another server running windows bare metal that serves as a secondary NAS for other things. And Truenas is the one that has given me so much more confidence that I cant really describe it. Data integrity is maintained by paranoid levels of paranoia. Where as on windows I am dependent on the HBA to do that. And its just not as inspiring of trust as Truenas Core has been. For my VM's I use a dedicated ESXi box (soon to be migrated to XCP-NG) which does use HBA Raid for its local store. That way Truenas has continuous control over all aspects of its hardware on its own machine. I always recommend running 2 computers if possible, because hybrid systems always need to compromise somewhere.
@_ClericalError_
@_ClericalError_ 2 жыл бұрын
I think you might have run into a limit in NTFS at 64TiB, you would need the "Workstation" or a server version of Windows to format the pool as ReFS in order to get above that limit. ReFS is supposed to compare relatively favorably with ZFS, but as I'm a FreeBSD/ZFS fanboy, I approve of your use of TrueNAS in this instance. ;)
@agy234
@agy234 2 жыл бұрын
How is Dave not running Enterprise already?!
@backupaddict1356
@backupaddict1356 2 жыл бұрын
REFS would have solved it all... and is far stable then truenas
@WAGISDev
@WAGISDev 2 жыл бұрын
Love that line..."...knowing something is possible, even if you then have to look up the actual steps later...". So true and man a project has that notion started.
@dumpsterdiverspcreclamation
@dumpsterdiverspcreclamation 2 жыл бұрын
Flashing controller cards can be a bit of a pain. I don't blame you for dodging this. I had to do this with an LSI card to get my system to run in IT mode because it was configured for UNRAID. I'm still a fan of RAID 10 and quite pleased with my current set up.
@MrREDSTAR20
@MrREDSTAR20 2 жыл бұрын
Hey Dave this is a really great channel I love your stories! Never stop making videos👍🏻
@snusmumriken232
@snusmumriken232 8 ай бұрын
Thanks for exercising Storage Spaces. I have been daydreaming about what it would be like to build a large array using it. Now I see!
@nathanmielke1977
@nathanmielke1977 2 жыл бұрын
This is a 500 lvl class and I just passed the 101 lvl..... Great teachers inspire students. Thank you
@fmh357
@fmh357 2 жыл бұрын
I feel like such a nerd. Being a field tech/engineer for most of my life I know what you're talking about but will never even get a sniff at this kind of hardware (retired) let alone be tweaking on it, yet here we are with me watching the whole video. Thanks by the way. How long have I been in the game? Remember 32K of core memory with 2K of cache?
@vertigo1055
@vertigo1055 2 жыл бұрын
Definitely deserves the Thumbs Up! Great knowledge on a new Hyper Visor that I had no idea existed. Although I don't need 400+ TBs of data storage I am doing calculations to get a Raid set up that will enable me to have around 40 to 50 TBs of storage available. My reasons are mostly due to 3D Model Texturing and the size of those textures in order to create eye popping materials. 2k, 4k, and 8k textures en masse take up a lot of space. I also like to hoard raw textures so that I can go back to them. I do however also compress using the .DDS format and will be storing those as well. To put things in perspective I currently store everything on a load of external drives piled up in a cardboard box. It's not the best solution :D. Cheers! Stay Healthy and Stay Sane!
@peterjansen4826
@peterjansen4826 2 жыл бұрын
"...Houston cautiously agreed to install Windows on it". I can imagine the guy of Houston telling his colleagues about it and everyone laughing out loud. 😁
@jimwilley3539
@jimwilley3539 2 жыл бұрын
Dave has style. Love the tech insights served up with a side of humour that would probs be lost on most people.
@bfth121
@bfth121 2 жыл бұрын
Good stuff Dave 👍 I decided to virtualise Freenas on my Windows 2019 box, there's a few powershell commands you can use to assign a HBA card to a Hyper-V VM, works great! Saves having to emulate both Windows and Freenas in Proxmox - I can just run Win Server and virtualise my NAS. I also UPS protected the box, with a script to gracefully shutdown the VMs and server when battery power reaches a critical level. Fun times!
@ddognine
@ddognine 2 жыл бұрын
A few years back, I was seriously into video production as well. So I completely felt your pain when describing why you need the storinator. For various reasons though, I am no longer doing video production, and it's a good thing too. It was close to a full time hobby. I am not autistic, but because I am an engineer, I had the aptitude to go down the endless rabbit holes associated with hardware and software. It was never ending to be honest, like I was chasing my tail, or in search of some holy grail that nobody, not even me, could accurately describe.
@123sleepygamer
@123sleepygamer Жыл бұрын
Meanwhile, me; >Takes an old HP with an AMD Athlon 2 X4 645, 18GB RAM, and a 'business class' ATI Radeon R7 250. Pile as many old 2TB HDDs in it as I can and SSD for OS. Slap OpenMediaVault 6 on it. Throw it behind the dresser. Profit. Thankfully, I just needed a simple DIY NAS for backups and being a media server. I don't even use Plex. I just run DLNA on OMV, works like a charm. You sir are the people that find those problems nobody else would, that would go unsolved otherwise. I don't have that level of patience, respect.
@mmaster23
@mmaster23 Жыл бұрын
Nice video Dave. Quick comment on the Storage space creation UI, it's not really the way to create it. You really need to break out the Powershell cmdlets in order to properly calculate and create the storage space you want. We shouldn't include the UI anymore, it's really bad.
@eukat3ch
@eukat3ch 2 жыл бұрын
Awesome again, also glad you hit up Jeff, he's really good at setting up these things.
@geehaf
@geehaf 2 жыл бұрын
Love this. 420TBs. I'm reminded of the days of installing MS Office from 30 odd floppy disks. How things have changed. The Storinator "I'll Bit Back...up".
@dmitrychernivetsky5876
@dmitrychernivetsky5876 2 жыл бұрын
Said, the stud finder makes it easy to find perfect level. Proceeds to drill the second hole half an inch below the perfect level line.
@kelvington4182
@kelvington4182 2 жыл бұрын
Nice! The mounting part of the video felt very "Red Green" ish! If only you had made it look like a B/W super 8 film! :)
@JeremyMcMahan
@JeremyMcMahan 2 жыл бұрын
Great video. It was fun to see the comparisons with Storage Spaces and TrueNAS... I'd love to see how you finalize things and how you set up replication, snapshots and backups too.
@TheCuttz1984
@TheCuttz1984 Жыл бұрын
I just want to say. I appreciate you and your knowledge. And can listen to you for hrs. Thanks for sharing 🙏
@AnnatarTheMaia
@AnnatarTheMaia Жыл бұрын
"IT mode" is the professional mode; it turns the buggy RAID controller into JBOD, so that it can be used and properly protected by ZFS.
@yourfriendwill
@yourfriendwill 2 жыл бұрын
never bought a LTT bottle but if there were a "Dave's Garage" equivalent I'd buy it day one, point to it often and say "yeah this dude coded the original task manager"
@DavesGarage
@DavesGarage 2 жыл бұрын
You can buy a mug :-)
@yourfriendwill
@yourfriendwill 2 жыл бұрын
@@DavesGarage done! I'd tag you but I suspect that's not a youtube thing, so just know that by the end of the month I'll be enjoying my coffee in your mug and explaining to both my cats - and anyone else who will listen, even odds on my wife - the significance of it. thanks man, love your stuff
@christophertstone
@christophertstone 2 жыл бұрын
Great video. Absolutely fascinating watching someone who dwarfs my knowledge in so many areas, just tip-toeing into storage. I've been doing NAS/SAN administration for about two decades, really interesting seeing what you think is worth sharing, or intimidating.
@afterthesmash
@afterthesmash 3 ай бұрын
"I wasn't really in the mood for flashing storage controllers ..." Ha ha. I think I've spoken those exact words in my past life.
@garbleduser
@garbleduser 2 жыл бұрын
@0:55 Nice Cristal D'Arques Durand Ruby Red Goblet.
@mikesveganlife4359
@mikesveganlife4359 2 жыл бұрын
I can share that long term storage spaces and JBODS can be a real bottle neck in an small office environment. Maybe it will work well for your usage, but we had constant performance issues when working with such a system when hosted by Rackspace. The biggest problem was random slow downs and a lack of expected performance. Various issues with HBA's, fiber cables, and even setting that Microsoft engineers helped with we never could get it stable. That was early storage spaces, hopefully the current version is better. Eventually we migrated to a new provider who privded virtualized storage on NetApp filers and unleashed the power we needed.
@muchosa1
@muchosa1 2 жыл бұрын
In 2000 I remember helping build Ford Motor Company's terabyte cluster while working for Digex. How technology advances so quickly.
@sonicalstudios
@sonicalstudios Жыл бұрын
Great Video. TrueNAS is amazing, I run it on bare metal and also with Proxmox on another server for my backups with identical drive setup each with separate 10GiB connections so the backup can happen at the same time as using the drives with no hassles
@ejharrop1416
@ejharrop1416 2 жыл бұрын
Thoroughly enjoyed watching and listening. I understand some and enjoy the rapid flow of information. Very cool. Thank you and cheers.
@midnightwatchman1
@midnightwatchman1 2 жыл бұрын
thanks, man I did exactly what you did. I am a windows server man but recently did a proxmox and 45drive implementation , it worked great but looking and look at videos I realise there much that could have gone wrong
@eugenezenzen
@eugenezenzen 2 жыл бұрын
Dave, I think I understood about 5-10% of this but enjoyed every minute! Thanks!
@DavesGarage
@DavesGarage 2 жыл бұрын
Glad you enjoyed it!
@R1ck3stR1ck
@R1ck3stR1ck 2 жыл бұрын
Well Dave, I saw the title and instantly thought of the “Petabyte of Storage” and you then mention it within 60 seconds of starting the video. Absolute legend 😁👍
@mrzvaniga3851
@mrzvaniga3851 2 жыл бұрын
I used to really LOVE windows,. as a tech geek from the 90s... but I moved to linux years ago and your video reminded my why :) I love your content Dave, keep on click'n!
@Stoney_Eagle
@Stoney_Eagle Жыл бұрын
I initially skipped this video because of "yet another 45 drives sponsored video" but I am actually glad having watched this. I now know the existence about something I need later 😊
@joejoezidane
@joejoezidane 2 жыл бұрын
And I feel in heaven with my 6tb Nas, and you came up with this insane storage.... Great video! It would be interesting to test the hardware with something like OpenNas, or Synology OS...
@kcox005
@kcox005 2 жыл бұрын
It's crazy that I'm doing the same things at my house I don't have the fiber speed as you do but my server has the sfps to do so...I plan on adding this later on, as I'm just a KZbin learner try to get a network built...great job lov ur videos
@MrAleOV
@MrAleOV 2 жыл бұрын
Hey Dave, have been watching since last summer. I love your videos! You have been an inspiration to learn about programming and technology of the past (I'm from 2000 so even 1990's technology is from the past for me). Congratulations on passing 250k subscribers. Hope to keep watching your content, keep making it great!
@ElJohnerino
@ElJohnerino 2 жыл бұрын
Your wall mount looks as super-level as some shelving I put up a few months ago. It's SO LEVEL I had a call from NASA asking WTF I'd done cos it was messing up some of their satellites. Good job, Dave. Levelness FTW.
@allanrichardson9081
@allanrichardson9081 2 жыл бұрын
All you need is ONE tiny (level 0.1) earthquake and you have to redo the whole project! Mother Earth has her faults, but when we dwell on them, it’s our fault. 😊😊😊
@dannydoolhoff4215
@dannydoolhoff4215 Жыл бұрын
@@allanrichardson9081 hahaha... faults - earthquake... 🙂
@NonTwinBrothers
@NonTwinBrothers 2 жыл бұрын
Omg the thumbnail change ...damn now I wish I saved the original
@BillBroadley
@BillBroadley 2 жыл бұрын
Nice build. A few comments for more general use. Don't use (or pay for) MS Windows, don't pay for expensive hardware RAID and then disable, don't pay for battery backed ram and disable, don't use and don't mount a rack vertically and try to blow hot air (that wants to rise) down. Do use ZFS, do use a HBA (no hardware RAID) adapters, do mount a rack mount to use the standard front to back or bottom to top airflow, and do add a SLOG/ZIL/write cache. Also keep in mind the virtualization only has low overhead for cases where you passthru the RAID adapters to the virtual machine. Fully virtualized systems (no pass-thru) generally have high I/O overhead, often not visible on large file transfers, but very visible in real world workloads that involved a mix of read/write and a mix of small and large files.
@uraniun235
@uraniun235 2 жыл бұрын
Dave, I love your videos, and this one especially speaks to me since I've run various RAIDs at home for over a decade now. My current setup is goofy and overcomplicated; I'm used to running a Windows server for file storage and other services at home, didn't feel like re-working that all to run on Linux servers, but still wanted to get away from hardware RAID (although that old Areca did serve me fairly well). My "solution" was to build a new server as a hypervisor; I spun up a Debian VM, passed through the motherboard's built-in HBA (reflashed to IT mode) to the Debian VM, and then used mdraid on Debian to create a RAID-6 storage pool (raw, no filesystem). I *then* spun up a Windows VM, and attached the Debian array to it via iSCSI, then formatted the array with NTFS. It works well and even proved to be expandable; when it came time to add another drive, after installing the drive I just ran the appropriate commands on Debian to expand the array, and then expanded the partition in Windows. The biggest challenge I faced was the initial iSCSI configuration, as some of the documentation was... not quite complete. I wouldn't recommend this setup to anyone else, and I'm thinking I'll go a different direction in the future; but it was kind of fun to setup. (In case anyone's wondering, I didn't want to go ZFS because I *really* like having the option to bump array capacity one drive at a time.)
@DavesGarage
@DavesGarage 2 жыл бұрын
That's very close to what I'm doing with TrueNAS on Proxmox with the HBAs and Optane passed through...
@iloveveggies7634
@iloveveggies7634 2 жыл бұрын
Cool video Dave. Proxmox runs on Debian but I get the idea that it’s a dedicated OS. It uses KVM, which means it allows VMs to execute code directly on the CPU. That’s why CPUs with IntelVT can run VMs at near native speed. You can also let them access their own RAM with the IO MMU support! Proxmox is really great for this and with the CPU you have, you can potentially pass the drives directly to TrueNAS. Good luck!
@zgelrevol9682
@zgelrevol9682 2 жыл бұрын
nice bolt on thinking - thank you!
@trueriver1950
@trueriver1950 2 жыл бұрын
Yes, it's a fork of Debian, strictly speaking, rather than "running on" Debian. There are mods to the kernel for example. It can be classed as a type 2 hypervisor in that periodic uses the KVM modules in the modified kernel to run other VMs.
@iloveveggies7634
@iloveveggies7634 2 жыл бұрын
@@trueriver1950 You can install KVM on Debian :) My point is that Type 1 vs Type 2 doesn't matter anymore since we have CPU extensions.
@trueriver1950
@trueriver1950 2 жыл бұрын
@@iloveveggies7634 and? I didn't dispute that. What I said it's that Proxmox is a fork of Debian, which it is. I really don't get your point...
@iloveveggies7634
@iloveveggies7634 2 жыл бұрын
@@trueriver1950 I'm sorry you are having issues getting my point. Sure Proxmox is a fork of Debian but the line between Type1 and Type2 hypervisors doesn't matter anymore.
@Theborg72
@Theborg72 Жыл бұрын
Thanks for this and so fun to see you using proxmox which is based on debian and also truenas scale which is also based on debian. There are times when you can trust linux over windows
@bluewombat
@bluewombat 2 жыл бұрын
Love the fact you reached out to Craft Computing, my two favorite creators working together, now you just need to work with Technotim too :-)
@MiniArts159
@MiniArts159 2 жыл бұрын
Glad to see ProxMox representation on the channel, but I find it a bit of a missed opportunity to not mention how it's basically just Debian Linux with some special sauce, and that the "Type-1 Hypervisor" in this case is actually just the Linux Kernel's KVM. Also highly reccomend looking into XCP-ng and Xen Orchestra at some point soon!
@DeeDee.Ranged
@DeeDee.Ranged 2 жыл бұрын
Seeing a CRT monitor of AOpen took me back 22 years when I used an AOpen AK72 motherboard and I might have it still laying around somewhere.
@JeevaDotNet
@JeevaDotNet 2 жыл бұрын
Installed 2x 500TB Dell R740xd2 at our DC last week. Got 12 PB Raw, running ceph. Also got about 50 x seagate 16TB SAS12gbs drives at home which are all dead. They all died outside Dells 1yr basic warranty, by one month. First one died literally on day 366.
@puckchew
@puckchew 2 жыл бұрын
day 366 at 00:00:01? 🤣
@soniclab-cnc
@soniclab-cnc 2 жыл бұрын
I have been so happy with proxmox. It really has made everything so easy and stable. I still have two bare metal TrueNAS servers for my data rather than running my storage virtualized. Proxmox runs absolutely everything else including 16 security cams, nextcloud, unifi ,plex, AirVideo, etc...
@Grenade71822
@Grenade71822 2 жыл бұрын
I would be interested to see if you can get both nics going at the same time on the truenas, then use SMB3 multi pathing to double your rates, and get a 2 port 10gb nic on your pc and see if you can get it to work. Love the video!
@OzTechGuy
@OzTechGuy 2 жыл бұрын
The progression of time has made seen astronomical changes to storage. My first PC (built from 3 'faulty'/scrapped IBM XT's) had a 10MB MFM hard drive, my 1st purchased PC had a 120MB IDE, I administered a fridge sized 1TB EMC Symmetrix array in the workplace when they were new and the 'ducks nuts' and nowdays I have tens of terabytes of SSD and spinning ruse storage in my desktop PC with yet more via NAS. I earned this grey hair :)
@robfti
@robfti 2 жыл бұрын
Love the bloopers at the end. Great video by the way.
@siberx4
@siberx4 2 жыл бұрын
Note that depending on your needs, Proxmox itself has "serviceable" support for ZFS built right in to the hypervisor operating system (notable, compared to many other bare-metal hypervisors out there that normally have very limited support for high-tech software-based storage layers without paying huge licensing fees). If you want a slick web-based GUI for managing your file shares for edge devices then passing the HBAs through to a NAS-specific operating system as you do here is the right way to go, but if you just want a server that reliably hosts some virtual machines and you do a bit of other stuff on the side, you can save an abstraction layer and keep the ZFS stuff in Proxmox, store your VMs on that pool and still use it for other things with a bit of command-line work.
@austinhiggs7257
@austinhiggs7257 2 жыл бұрын
Great video! Also liked the outtakes - keep `em coming
@blahorgaslisk7763
@blahorgaslisk7763 2 жыл бұрын
A way to long time ago, or at least it feels like that right now, I was testing a early prerelease version of a Adaptec Raid controller. Using three of them and I think it was 24 drives, could have been more, I created three R0 stripes, one on each controller, and then striped those in windows. I remember that we managed to just touch 3GB/s. Our contact at Adaptec just laughed in the phone when I sent him the results. Totally impractical but it was a treat to see the numbers. I did discover some problems with the drives reporting long latency which had Adaptec send a tech with diagnostic equipment to diagnose the problem. Turned out to be hardware related so they had to make a revised version of the controller silicon. It delayed the introduction, but better that than shipping a unreliable RAID controller. Now I'm not a storage specialist but the tests I did stressed the cards in ways that regular benchmarking and testing software didn't. At the time it was very hush hush and I was not allowed to talk about it. I did learn that at first they'd thought it was an isolated occurrence related to the drives and firmware as no matter how much they stressed the cards in the lab they couldn't replicate the problem. It wasn't until another company reported similar problems using other HDD's that they kicked the problem up the ladder and sent out lab technicians with test equipment to fly to Europe and grab the raw data...
@transmitterguy478
@transmitterguy478 2 жыл бұрын
Great video Dave, as always. You are a great big shiny, hot-swappable, TO-3 transistor, in my eyes. LOL😂
@michaelpezzulo4413
@michaelpezzulo4413 2 жыл бұрын
I thought I was cool when I plugged a 250gb flash drive into my raspberry pi 4. Congrats Dave, good job.
@tylercgarrison
@tylercgarrison 2 жыл бұрын
"LTT fat water bottle budget" xD love LTT and you, and Jeff, and other Jeff, and all you techy youtubers. also shoutout to Chris at explaining computers. You are all amazing
@gorinator
@gorinator 2 жыл бұрын
Dave, it's time to treat yourself to an ultra-quiet California Air Tools compressor! A game changer. Runs and sounds like the compressor in your refrigerator.
@Ashimo
@Ashimo 2 жыл бұрын
Love seeing the ocz revo. Mines been a workhorse as well.
@x1625
@x1625 2 жыл бұрын
Wow! Nice real glass version of the Dolorama Halloween wine goblet.
@forthrightnight
@forthrightnight 2 жыл бұрын
First time viewer. One minute in; liked and subscribed. Can't wait to dive into your library. Back to the vid.
@IEnjoyCreatingVideos
@IEnjoyCreatingVideos 2 жыл бұрын
Nice video Dave! Thank you for sharing it with us!💖😎👍JP
@mtech1961
@mtech1961 2 жыл бұрын
I am surprised you put all that mass on two screws? You are Brave.
@markroster8395
@markroster8395 2 жыл бұрын
Now all I need is a Way-Back machine in order to take this back to 1972 and become Zar of the World. Great video!
@3sotErik
@3sotErik Жыл бұрын
I really do love listening to your videos. I'm curious how much time you spend preparing them. The way you loop back to previous points indicates that you are definitely scripting them to some degree and I'm curious, do you make bullet points or do you write a full-on script? Thank you & keep it up. I've only come across one of your long-format videos so far and loved it, except for all of the ads that aren't in the shorter 25 minute videos.
@erichkohl9317
@erichkohl9317 2 жыл бұрын
I'll never forget micromanaging the space on my 20 MB hard drive in my first Turbo XT clone.
@user-hd7wd4nu1o
@user-hd7wd4nu1o 2 жыл бұрын
Yea I remember "Stacker" using on my HUGE 10MB drive to get an extra 5-6 MB!!! now my canon camera produces 15MB files per pictures :)
@puckchew
@puckchew 2 жыл бұрын
by micro managing did you mean that you were copying the "essential" and frequently used MS-DOS files one by one to make sure they reside on the front most disk part? and set read only, hidden, system (where possible) attributes to make sure defrag won't move them
@erichkohl9317
@erichkohl9317 2 жыл бұрын
@@puckchew heh no, I never went that far, I was basically referring to the constant process of deciding which programs stay and which programs go.
@puckchew
@puckchew 2 жыл бұрын
@@erichkohl9317 i was being serious though. i did that 🤣OCD
@erichkohl9317
@erichkohl9317 2 жыл бұрын
@@puckchew Haha, don't feel bad, I can be the same way sometimes!
@browntigerus
@browntigerus 2 жыл бұрын
Great video Dave, I dumped storage spaces and switched to TrueNas. Drives run nicely. Still working out spindowns unfortunately. I don't use my HD array all the time. Once a day, and I want WD 7200 spindown after 30 min of no use.
@juhojohansson1716
@juhojohansson1716 2 жыл бұрын
Thank you for yet another inspiring and educational video. It really is nice to learn about things that are possible with computers. Made me imagine using just one computer for an entire team... Might make maintaining the whole system a bit easier. At least it would make really quiet office, if one could put all noise generating hardware in a closet... But that idea would require still much studying, planning and so on... But even if I would never do any of that, it is still nice to have new ideas to enertain ones mind with.
@TrevorNewsome
@TrevorNewsome 2 жыл бұрын
Love the wine glass, it only needs a Daves Garage logo on it and then Im sure it will be a big seller :P
@esrevinu.
@esrevinu. 2 жыл бұрын
this was an excellent and informative video, or new and old alike. Well done, sir! liked and subbed.
@pierQRzt180
@pierQRzt180 2 ай бұрын
I like the idea of pushing systems where they are not supposed to go.
@Basement-Science
@Basement-Science 2 жыл бұрын
Windows Storage Spaces are a buggy mess, especially if you use the GUI for it. The powershell cmdlets for it at least have a few less bugs. If you use it, at least set it up once and then never change it without making a full backup first. If you do the "wrong" thing, it will absolutely corrupt your file system. Anything other than a stripe set is super slow as well.
@bird175
@bird175 2 жыл бұрын
Maybe I didn't see it, dust screen in front of first fans? It's looks like good air flow, and I'm sure Daves house us super clean.
@anon_y_mousse
@anon_y_mousse 2 жыл бұрын
Good job, even if you did push for Windows. I love the hanging mount, saves a lot of space in the closet.
@ericmiller4784
@ericmiller4784 2 жыл бұрын
That green cable kinked over like that is driving my OCD tendencies into orbit!!
@DavesGarage
@DavesGarage 2 жыл бұрын
You should watch my 10 gigabit video! I have a whole segment on that stuff.
@JPEaglesandKatz
@JPEaglesandKatz 2 жыл бұрын
I'm drewling a bit too much almost to comment but wow that is a nice system.. And glad you finally came to your senses :) and used truenas and ZFS!!! :) Don't even want to know what the total cost of that system is... obviously wayyyy over the budget from most individuals.. Have fun with it and thanks for sharing! Oh and do not ever forget to safely backup your ZFS encryption keys... Losing them is losing all the data.
@BigBoxLittleBox
@BigBoxLittleBox 2 жыл бұрын
Storage Spaces for a MIRROR configuration works well and performance is pretty good (50% capacity overhead though). I have tried a number of Storage Spaces PARITY configurations and found that that the performance (especially write speed) were terrible. I had much better performance using a traditional hardware RAID card (with a battery backed write cache module) for RAID5 configurations. I have since moved on to using UNRAID, which allows for creating disk arrays using different sized disks, has great application support (using Docker) and also allows disks to be spun down/spun up based upon disk activity. As a home based NAS - the disk spin down/spin up saves power and reduces noise levels. I do have multiple SSDs configured as "cache" drives to run Docker Applications, Virtual Machines and support 2.5 Gb/s wire speeds for SMB access. Downsides to UNRAID is that native write disk performance isn't great and it also takes many hours to "zero" a disk if you want to expand/shrink the array WITHOUT having to regenerate parity across the whole array. For my use as a home NAS/Media Server UNRAID has been significantly easier to manage and maintain than a Windows based NAS Server. The next version of UNRAID is due to have native ZFS support available within the GUI (at present it's either command line or can be achieved with a "hybrid" mode of managing via a TrueNAS VM and then exporting/importing the ZFS configuration).
@rakly347
@rakly347 2 жыл бұрын
"Their phat water-bottle-budget" I lolled so hard!
@nathanielmoore87
@nathanielmoore87 2 жыл бұрын
Great video, Dave!! Also love that old CRT you are rocking in your utility closet.
@eDoc2020
@eDoc2020 2 жыл бұрын
From the precise screen borders, the sheen of the display, and the size of the stand I can conclude it is actually an LCD monitor. Although at first glance I also thought it looked CRT-ish.
@milk-it
@milk-it 2 жыл бұрын
Outtakes like at the end of Cannonball Run, love it!
Bill Gates' Easter Egg!
23:39
Dave's Garage
Рет қаралды 219 М.
Storage Media Life Expectancy: SSDs, HDDs & More!
18:18
ExplainingComputers
Рет қаралды 507 М.
I built a computer that CAN'T break - Proxmox Clustering
18:49
Linus Tech Tips
Рет қаралды 1,3 МЛН
How RAID Works
19:46
Dave's Garage
Рет қаралды 81 М.
New Windows Easter Egg Discovered - And I'm in it!
10:23
Dave's Garage
Рет қаралды 181 М.
Running VMs in TrueNAS Scale - Should you run this instead of Proxmox?
21:05
EXPOSED: The Windows Rootkit Scandal by Sony
20:47
Dave's Garage
Рет қаралды 414 М.
Synology vs UGREEN - Who makes the BEST NAS??
19:52
Raid Owl
Рет қаралды 125 М.
So if Hardware RAID is dead... then what?
29:00
Level1Techs
Рет қаралды 156 М.
08.Secret History of Microsoft Gary
16:42
Dave's Garage
Рет қаралды 116 М.
Windows Tales: Task Manager, Pinball, ZIPFolders, and More!
41:30
Dave's Garage
Рет қаралды 162 М.
Everything you SHOULD know about your Computer but don't!
19:59
Dave's Garage
Рет қаралды 476 М.