How Disk Arrays and Parity work - Raid and Unraid

[@SpaceinvaderOne]

I should have said that raid 5 can be expanded (although this does depend on the controller) . But it is risky as you cant expand a raid 5 array without loosing fault tolerance. A rebuild is triggered when expanding raid 5 whilst it redistributes all the data across the disks. Until its finished if a disk fails the array would be lost. Expanding the Unraid array doesn;t require a rebuild or loosing parity. Same with adding a second parity drive.

[@i_am_macgyver84]

I just want to say thank you for the great work you do here. It was probably Linus Tech Tips that first introduced me to Unraid doing these crazy fun yet unrealistic projects, but It has been your channel that sold me on Unraid and has helped and guided me so much a long the way.

[@aliens1990990]

same ill agree on this - dont get me wrong ive had some hiccups with a couple of dockers losing their permissions so databases wouldnt be read but generally its working perfect just need to get my backup solution sorted

[@julcrimson3323]

I wouldn't say the video is long, I'd say it's full! You're a real source of information and a total enabler in the sense that you enabled me to understand and do many things with unraid I never could have done otherwise. Thanks Ed! The timing on this video was great for me, I just upgraded to dual parity and wasn't quite sure how it works,

[@SpaceinvaderOne]

Great thanks Jul and thanks for watching :)

[@i_am_macgyver84]

While there are plenty of applications a Raid setup is better. I did a lot of research before choosing Unraid and videos like this confirm that I made the correct choice for my needs. I started off almost 2 years with 3 - 8TB ironwolf's (2 Parity 1 Data) and a 250gig ssd cache drive. Now I'm up to 10 - 8TB drives including the 2 parity with a mix of ironwolf, red plus, and n300 and a 1TB cache drive. I have enough room for 3 more drives in my case so I'm begining to plan ahead of time the options if I need to expand farther.

[@johngaltline9933]

Not so much for you as it doesn't matter any longer, but for other folk, there is almost no reason to ever use two parity drives and one data drive. In the very niche that you have a server that is inaccessible to replace failed drives, it might make sense, but other than that you don't get any benefit out of it if you plan to replace drives if and when they actually fail and not just leave the server with a bad drive indefinitely. Outside of that niche, you're far better off going with one parity and two data drives, or one parity, one data, and spending the extra money on cache drive(s) or better hardware elsewhere in the server. Lastly, while not much of an issue for any modern CPU, for folks using cheap, old, enterprise gear, the processor has to work much harder on the calculations on the second parity then on the first. no point in wasting the power or the cores computing parity2 when it doesn't do anything for you.

[@i_am_macgyver84]

@@johngaltline9933 I get what you're saying, I started my build off knowing i'd be expanding. I now have 2 - 8tb parity drives and 15 - 8tb storage drives for a current maximum of 120tb. I even had to make an external setup to hold extra hard drives because my case is full. I'm using an old amd fx 8250 cpu, I would love to upgrade it but I have to wait a little longer. But when the time comes unraid makes it very easy to migrate.

[@johngaltline9933]

@@i_am_macgyver84 I started out with a little “shelf” I made to hold the disks in my first version of my server. Always had about 29 drives due to using a bunch of used 500gb drives I got cheap. The current version uses a super micro board with dual opteron processors. I’m using two 4U server chassis that hold 15 drives each. I raid motherboard in one and my pfsense board in the other. Works really well except if a drive fails it is a hassle to change them. Highly recommend getting a case or cases with hot swap sleds if you can afford it. Currently have all bays full, mostly with 2 and 4 tb drives

[@Megabitus98]

Just recently i was trying to find a way to explain to my friends how Unraid party works. This came perfectly. And amazingly done as always, thank you for all the work you done for the Unraid community!

[@SpaceinvaderOne]

Your welcome. Im glad you enjoyed the video :)

[@Bertman_5429]

I need to setup two drives one to copy the other. Do I set up both drives as disk 1 and 2 or do I do a Parity drive and disk one?

[@i_am_macgyver84]

I'm pretty sure it's spaceinvader who has a video from a bunch of years ago explaining parity, but he went into even more depth in this video. The full depth is beyond me, but videos like this help myself to understand it more.

[@ndandan1369]

🎯 Key points for quick navigation: 00:00 *📚 Introduction to disk arrays and parity for beginners.* 00:22 *🔍 Overview of parity and its role in Unraid arrays.* 00:50 *📖 Definition of parity in computing: checking binary values for errors.* 01:20 *💾 Explanation of binary data transmission and error checking using parity bits.* 02:29 *🖥️ How parity bits ensure data integrity in binary messages.* 03:13 *🔢 Example of using even parity to check data integrity.* 04:20 *⚠️ Parity bits detect errors but can't correct them; need for retransmission.* 05:44 *🔄 Introduction to parity disks for redundancy in Unraid arrays.* 06:12 *💽 Parity disk stores data to ensure even parity across sectors.* 08:42 *💡 Using parity to recover data from a failed disk in an Unraid array.* 10:35 *🔗 Comparison between Unraid and RAID 5 arrays.* 11:02 *🗂️ RAID 5 stripes parity and data across all disks, requiring uniform disk sizes.* 12:41 *🔄 Adding disks to an Unraid array without affecting parity.* 13:52 *🛠️ Using mixed-size disks in Unraid arrays without affecting parity.* 15:58 *⚡ Unraid arrays use less electricity than RAID 5 due to selective disk spin-up.* 17:22 *💡 Uneven wear on disks in Unraid arrays reduces simultaneous disk failures.* 18:46 *🛡️ Unraid allows adding extra data disks and dual parity for added protection.* 19:44 *💨 RAID 5 offers faster read and write speeds compared to Unraid.* 20:11 *🏎️ Use of SSD cache pools in Unraid to improve write speeds.* 21:48 *🖥️ ZFS pools rumored to be coming to Unraid, offering new storage options.* 22:01 *💡 Explanation of dual parity in Unraid: P drive and Q drive using different algorithms.* 22:56 *💡 Explanation of bitwise operations using a truth table.* 23:24 *🧩 Understanding bitwise OR operation through practical examples.* 24:04 *🔄 Introduction to bitwise XOR and how it differs from OR.* 24:32 *⚙️ Exclusive OR (XOR) requires only one condition to be true.* 24:47 *🔍 Applying XOR to parity disk calculation in Unraid arrays.* 25:01 *📏 Example of XOR algorithm for two disks, demonstrating parity calculation.* 25:15 *🖥️ Extending XOR to multiple disks in an array for parity calculation.* 25:44 *🧮 Final parity result using XOR for multiple disks in Unraid.* 25:58 *📚 Summary of how XOR algorithm is used in parity calculation.* 26:11 *🎓 Importance of understanding parity for shrinking Unraid arrays.* 26:26 *👍 Encouragement to like, subscribe, and share the video for more content.* Made with HARPA AI

[@xGERKENATORx]

I keep finding myself looking at your videos over the years. Thanks for the legwork documenting all this stuff.

Great video!!! btw, 25:15, there is simpler calculation for this: 1+1+0+1 = 111, take the last number and we have "1"

[@sebarem0]

Love the detail at 2:47 with the Unraid logo, I didn't realize it was getting bigger and moving up the whole time until it was pushed down xD

[@hellhole7422]

Just getting into Unraid with my old gaming pc and some old hdds and ssds. Looking to upgrade my Synology 918+ and I needed to wrap my head around the parity disk and this helped a lot - Cheers!

[@roddybrod]

Didn't realize a second parity disk wasn't just a copy in case P1 fails, good to know going forward. And I suppose I should look for an actual internally rated drive to replace the drive instead of the shucked one in there. Thanks as always, I'm very new to unraid so these core beginner tips are always appreciated.

[@SpaceinvaderOne]

Your welcome thanks for watching :)

[@moritzwagner4332]

This is very interesting, thank you for making this video, would have never imagined someone doing videos about such specific stuff.

[@playati24]

Extremely helpful video (per usual). I'm sure I'm not alone in conflating the parity disk function in Unraid vs. RAID, and this changes the way I will approach hardware planning. Very much looking forward to the array shrink video - I have a disk that has started throwing read errors that I need to remove, but have struggled to Unbalance.

[@00Klingon]

Good video! A great primer for those who may be confused about it. It might be helpful to add a video explaining the differences between parity and backup as well as I know lots of new users seem to be confused about that.

[@drumbangerproduction]

I have a 4tb NVME and a 4tb enterprise hdd. I want to have the hdd mirror the NVME. With unraid should I make a raid1 with the 2 drives, or use the nvme as a cache but keep the files on there, or is there another solution?

[@dmynerd78]

Talk about timing one of my classes just covered parity yesterday! Watching this video really helped solidify those concepts so thank you! :D

[@SpaceinvaderOne]

Your welcome. Thanks for watching :)

[@johngaltline9933]

I like the long format of this video. How about a part 2 for the nerds? How parity drive 1 works is simple enough and is explained everywhere. I want a deep dive on parity 2. No one ever explains it at all.

[@patrickdoty5534]

i HONESTLY DON'T KNOW WHAT I WOULD DO WITHOUT SPACEINVADER ONE!!!!!!!!!!

[@wedemandcookies]

Wonderful stuff! I had a basic understanding but actually learned a lot, thank you! :)

[@SpaceinvaderOne]

Glad you liked it :)

[@amateurwizard]

Nice video, very informative especially for the people that don't read the wiki before incorrectly associating the kind of parity unRAID does to others.

[@SpaceinvaderOne]

Glad you enjoyed it!. thanks for watching :)

[@kinesis]

Very informative & useful video. Great work!

[@try-that]

I don't use unraid, but I learnt a lot from video. Nicely done.

[@SpaceinvaderOne]

Thanks trythat. Glad you enjoyed it :)

[@soulivan827]

This video is wonderful, thank you.

[@KevinMarchese]

Watching Space Invaders video on a VM he helped make possible, Kudos. Listen to this man.

[@SpaceinvaderOne]

Thnankyou Kevin and thanks for watching :)

[@johnshort8378]

Great explanation. Thanks.

[@HelloHelloXD]

Great video. Could you do a similar video about ZFS (Z1, Z2, Z3) vs Unraid? Also could you make a video about protecting files that will never change (movies and photos)?

[@swoopedify]

Great video thank you for the explanation!

[@39zack]

If I have understood it correct, when zfs is coming to unraid, its for its own pools and not replacing the unraid raid pool

[@mpxz999]

I love you Thank you for being a great teacher!

[@IronCondorSimulations]

How often should we do a parity check? Great video! Thank you!

[@destronger5313]

You should be fine with once a month. Have be done during a time you’re not using it like when you’re sleeping. There’s an app for unraid called ‘Parody Check Tuning’ that you can spread out the parody check throughout the month.

[@moritz22]

How frequently should you do a Parity-Check?

[@SpaceinvaderOne]

I would say every 4 to 6 weeks.

[@girl-with-many-questions2659]

Who can name a handful of other software packages that offer "all parity on 1 disk" possibilities? It's a very clever method, but is UnRaid the ONLY choice? I love the "all parity on 1 disk", the "add disks at random", the "remove disks at random", the "mix any size/speed disks", the "spin down when not in use", and all the other great features. Why haven't DOZENS of other companies used those features?

[@SpaceinvaderOne]

Yes there are other solutions which have similar features. Take a look at Snapraid. Thanks for watching :)

[@songworks17]

Great video. Been wondering what the actual differences between RAID5 & Unraid was. Thanks!

[@SpaceinvaderOne]

Thanks for watching Stefan.

[@knex1312]

Perfect!!! Thank you!

[@IEnjoyCreatingVideos]

Great video Ed! Thank you for sharing it with us!💖👍😎JP

[@trp]

Great explanation thank you

[@h.b.7190]

What’s the song name at 9:49 ? Thank you and good video. Learned a lot today

[@Galaxywalkers]

Another Great video. Have to say thanks for all the videos you make on Unraid. I've learned a lot over the years from your videos. Just wonder if you have a video on configuring iSCSI on unraid? That works with Unraid Version: 6.9.2 ?

[@bjaurelio]

Great explanation of how Unraid's parity works

[@reoproedros]

can i use a partition of the bigger drive as parity drive and the rest of the drive as cache?

[@Dr-AK]

This is very helpful understanding Unraid vs raid, thank you. What can we do with Zfs now that Unraid has it? should cache pool be zfs?

[@Yeven.]

Great video, I had a question that may be dumb. With Unraid, If you have a 16tb parity drive and say (4) other 8tb drives. Does that ONLY give you access to upgrade the others later to up to 16tb while protecting just 1 failure at a time? Or could the single 16tb disk parity drive it could cover a possible 2 drive failure?

[@StreamingF1ydave]

Does dual parity require 2 identical sized drives plus a matching size in the array?

[@NickelCityPixels]

Thanks! I get it now! :)

[@IllMethods]

I watched this video a while back and everything made sense, but now I have gone to dual parity and am wondering how having two parity drives works... I have asked Google, but I don't really seem to be able to find an answer, and that seems to be the same for everyone else that has asked this question. Do you have a video for this?

[@MGTEKNS]

I've been going back and forth new videos and old videos because I'm rebuilding my unraid server. So far I know for sure to use my nvme drive a cache drive but even though I understand this concept you're explaining I'm not sure 1) what size the parity drive needs to be based on my environment and 2) if I'm using raid 1 in a pool does it even matter or is the parity drive just another share drive independently for a safer redundancy.

[@defyiant]

Can a cache ssd pool be 2 nvme drives?

[@Bertman_5429]

I need to setup two drives one to copy the other. Do I set up both drives as disk 1 and 2 or do I do a Parity drive and disk one?

[@viq234]

One of my drives failed and i dont plan to replace it as my other drive has space. Also i am transplanting to new pc because i had to give original build to sister for projects. Now im ready to build new machine, whats the step to ensure my ip address isnt in conflict as my old one was 192.168.45, do i try to assigng the same ip address on the router or do i just update unraids ip address because i remember telling unraid what the ip address was when made the usb. Thanks

[@nicktayloriv310]

Wasn't a "long" video in my opinion. I actually skipped back a couple times. When it's the info I'm looking for there's no such thing as a long video. 👍🏽👍🏽

[@dakid1102]

Thank you sir

[@LUCASINO1993]

Sry my english is not very good but do i understand that right, when 1 Disk fails the Parity cant restore all that data to a new disk, it only looks for errors?

[@dnwheeler]

If a single drive fails, Unraid can emulate the drive and recreate all the data. Once a replacement drive is installed, Unraid will write all the emulated data to the replacement drive, restoring the array.

[@auvers]

thank you for the fantastic videos that I have watch religiously over the last few weeks while setting up my Unraid server, I hope this isn't asking too much but is there any chance you could do an updated video on rclone? as I cannot wrap my head around mounting a Google drive for Plex, I am coming over from Windows and cannot figure out these damn config files

[@Techie4life]

Can you make a tutorial installing pihole with unbound please

[@YannMetalhead]

Good video!

[@robertt9342]

One of the reasons I went with Unraid was the limited loss of data in case of a fail.

[@tylerrobb]

Anyone out there running shucked drives as their parity disk? I was just about to build my first unRAID server and use a shucked drive as my parity drive... Now I'm reconsidering!

[@tbut7633]

I was until a few days ago. I became aware of the difference of SMR vs CMR drives and my shucked drives were SMR so I've bought drives to swap out my parity and half of my array (the half that is most commonly accessed, not the movies & TV). It's pretty expensive to buy quality drives. I've run my unraid for 12 months on SMR drives without any issues, yet. But, like I said, I'm trying to swap out the most vulnerable discs. I made and use the SMR drives in a backup server for weekly backups. I'm no expert! Probably just a few weeks ahead of you.

[@mike95826]

I built my server using the two best largest "red" drives as the parity drives. Then since I noticed that the disks fill up using disk 1 then 2 etc. I put my next newest which is also a "red" drive as disk 1. Then followed that pattern with some shucked drives at the bottom. Using my oldest shucked drives I created a RAID 1 pool to be used as a scratch pad (not as a read/write cache) where I keep files that I update regularly or files that can be easily restored if they were lost. As I upgrade drives in the unraid array (oldest first), I push that drive onto the scratch pad pool. Then the drive that comes out goes into my development server...the one with the oldest motherboard, power supply and smallest memory that is ok if it fails for an extended period of time.

[@MyOnlyResort]

Hey spaceinvader! Would it be possible for you to create a video on how to use fail2ban with nginx proxy manager on unraid?

[@kristof9497]

Tthank you.

[@Banner1986]

RAID 5 can be expanded, as well as converted to RAID 6, been available at least since the SATA 2 days

[@SpaceinvaderOne]

Yes sorry that is true. (but can depend on the controller) What i should have said is it isn't possible to expand without loosing fault tolerance. A rebuild is triggered when expanding raid 5 whilst it redistributes all the data across the disks, risking loosing the array if a disk failure occured during this time. Expanding the Unraid array or adding a second parity drive.doesn't require a rebuild or loosing parity.

[@Banner1986]

@@SpaceinvaderOne I can get onboard with that. I know back when I first started in IT, we were at the tail end of the scsi 320 generation, and even those controllers (old school LSI, before avago and eventually broadcom) could do raid expansion, though raid 6 was beyond their reach. To me, the biggest win for unraid is its efficiency - throwing in the checksum plugin, and I don't feel like it can be beat... as long as you dont require high performance. And once zfs pools become a thing that can be UI managed? 4 pcie 3.0 nvme drives as a cache pool pulling

[@miquelfire]

I seen hard drives fail in the same was as pictured in the thumbnail.

[@Santi-fe5mw]

Just point out a RAID5 can be expanded with more disks... linux software raid mdadm has de --grow flag, most hardware raid controllers allow this and BTRFS RAID with parity (5 or 6) is declared unstable by the developers so it's a big no if you care a bit about your data. The rest of the video is great even though you leave out some details about performance and parity checking or what happens if a disk gets corrupt and spits out corrupt data (with an XOR you can detect 1 error but cannot know which one is wrong or if there are more than 1 error) but seeing you thought it was a long video, I can understand why.

[@SpaceinvaderOne]

Yes btrfs parity raid is not a good idea. And sorry yes raid 5 is expandable. What i should have said it is not expandable without a rebuild during which time there is no fault tolerance so the data is at risk and Unraid array not needing to rebuild during expansion. Thanks for watching :)

[@StrawDragon]

Incorrect details on RAID 5 and adding drives as well as total storage with mixed drives...

[@girl-with-many-questions2659]

Every time I think I've found a video that fully explains "how can 2 parity drives possibly work", it does not. ;-(

[@SpaceinvaderOne]

Yeah i left the explaination at "it uses Reed Solomon "as the algortith is far more complex than xor. Thanks for watching :)

[@dnwheeler]

I think your question arises because the second "parity" drive (Q) isn't really a parity drive. In a literal sense, 2 parity drives can't work. Unraid uses one parity drive (P) and one Reed-Solomon drive (Q) that, when combined, can reconstruct all the data when any two drives fail. The details are not really important and are too complicated for a beginner KZbin video - take a look at the Wikipedia article and you'll see that it's definitely University-level mathematics.

[@F4ilu2e]

Hi. Your unraid content has saved me ton of times and always your videos come in superior timing. When im after someting unraid related, you have created an beautiful, easy to understand videofor me to follow on. However i have an request which i would really like and hope you to fulfill. Google is suspending free g suite accounts created before 2012, i have used my domain in gmail for decade, now google is planning to make this monthly subscription fee and im not up to it. Im already paying for youtube premium. So my question is if you could make a video setting up an mail server with your own domain name, secured and fully functional. I would love to donate for you, im shamed i havnt already (im poor), but this would trigger the donate button (not a huge amount).

[@sshomelab]

I never understood why people swear by 'conventional' RAID systems. To me it feels like it has way too many downsides to it. Expandability is very cumbersome and has a high cost to it, since you need to buy and set up a whole new RAID in 1 go. If something fails, a drive or your RAID controller, you lose everything. Your drives will likely fail close to eachother, so the chance of an unrecoverable error is greater. It feels that is has way too many downsides, compared to the upsides. What do I miss? Why is this so popular? I can somewhat understand the use of RAID 0, 1 and 10 in some usecases, but 5 and 6 are beyond me.

[@dnwheeler]

The most common reason I see is speed. Secondly, most traditional RAID arrays have their own processors and appear to the host computer as a single drive (i.e., the computer/OS don't have to be aware of the RAID array - it just looks like a single big drive). Software RAID sort of cancels that benefit (and some of the speed benefits). And finally, all the "cool" kids think RAID is the only "real" array, without even bothering to understand the alternatives.

[@derfacecrafter1869]

That Platter doesn't look good.

[@SpaceinvaderOne]

lol yeah its seen better days :)

[@derfacecrafter1869]

@@SpaceinvaderOne you skip exactly that part of unraid, I want to know how it work. The 2nd parity drive. But that's the only thing I don't like about this video. Very good work. I'll sent it to my cousin cause I wasn't able to tell him last month, how unraid work. Is it able to create a pool that have, like the array, a parity drive and data drives?