All of our data is GONE!

We experienced a MASSIVE RAID failure on Whonnock server and lost a crapload of data. Can we get it back?
Rackspace has your Dedicated Environments covered. Head over to www.rackspace.c... to learn more!
Data recovery done by www.WeRecoverDa..., thanks for the help!!
Pricing & discussion: linustechtips.c...
Support us: linustechtips.c...
Join our community forum: bit.ly/ZkLvE7
/ linustech
/ linustech
Intro Screen Music Credit:
Title: Laszlo - Supernova
Video Link: • [Electro] - Laszlo - S...
iTunes Download Link: itunes.apple.c...
Artist Link: / laszlomusic
Outro Screen Music Credit: Approaching Nirvana - Sugar High / approachingnirvana
Sound effects provided by www.freesfx.co....

[@derekroberts1693]

"We should physically print all of our videos, every frame... on paper..." This comment didn't get enough credit in the video. +1 props.

[@block0ee]

In 8k resolution

[@kenivia9476]

*house burns down*

[@memegangster2575]

*500 billon houses burns down U KILLED EVERYONE EVEN ME WHY DID U DO THAT !!!!!!!!!!!!!!!!

[@yazanarmoush]

Even the slow mo guys? 😂

[@vincusstreams]

with 1000fps

[@eukyre]

"How do you fix a server?" *"Go tell Linus"* this man's logic is genius

[@darkendrandomvids]

ah yes, genius.

[@whiskeyandchoke]

he'll have plenty of tech tips for you

[@iankelly8827]

I know its horrible to say but that did make me giggle a bit. It's what I would say

[@ixp8605]

Remember, You Only Liao Once

[@Nitroblast]

@@ixp8605 ayy

[@RCShufty]

I've been working in IT for 15 years and I can relate to the emotional rollercoaster he's experiencing.

[@jopett1]

Oh yes, I've been it in 3 years and I can relate.

[@Neroese1]

Oh, how many times...

[@AndyMorris_BAP]

Same....had a co-worker pull two 'predicted failure' drives out of an array you CANNOT pull two drives out at once....he left right after replacing both drives saying one was taking a while to rebuild. I quickly put the puzzle together when people started coming to let me know they couldn't access data any longer. Erased our public share drive and domain controller, was there until 6am the next morning rebuilding the domain from scratch because the backups he was in charge of were missing. He claimed it was a virus and got promoted to manager after that.....luckily he didn't last much longer after that.

[@RCShufty]

@@AndyMorris_BAP man, that sucks.

[@raptorx9379]

Andy Morris Just reading your exp itself increased my bp.

[@streax8885]

You Only Liao Once hits different now because of his sign Live Laugh Liao

[@felixbeckmann6459]

I like to think this that event implanted the idea into his brain

[@beefnoodlegaming]

_Nice to see that I'm not the only one who thought this_

[@Dimethro]

@@beefnoodlegaming same

[@JaydevRaol]

Yeah 😂

[@NFSmikaNFS]

Omg its the first thing i had to think of hahahagah

[@mejakoba6471]

Love how he calls tech support and he's like: 'Yeah this is Linus'

[@looneyburgmusic]

Most likely a small tech support outfit and he has to call them a LOT.... That's not really a good thing.

[@seanpe8474]

@@looneyburgmusic You probably missed completely the context of the video. It's definitely not easy setting up a RAID server, let alone diagnosing the problem and fixing it yourself. Linus is very much an IT professional, but he's not a server management expert although he is very knowledgeable. Your safest bet is definitely to call people who do this for a living, full-time, than to try and do everything 100% yourself (although Linus certainly did a lot on his own).

[@yoshi0k262]

@@seanpe8474 you missed the point if looneys reply

[@yoshi0k262]

@@gumbymofugga lol i never gave my opinion on what he said i just said he didnt understand what the other guy was saying

[@yoshi0k262]

@@gumbymofugga lol i never gave my opinion on what he said i just said he didnt understand what the other guy was saying

[@Nostalgia_Realm]

This still beats all reality shows I've ever seen on TV. The suspense is real!

[@fynkozari9271]

My 1tb seagate hdd failed too, but I managed to move everything off before it died.. Hard drives give u warning before they die.

[@cloudyyy-._.-999]

@@fynkozari9271 they do? does it say it in windows?

[@fynkozari9271]

@@cloudyyy-._.-999 NO when the hard drive makes a ticking sound its dying. My HDD wouldnt let me boot, even when OS in a different drive. So I removed the seagate drive and its fixed.

[@uglybad4]

@@fynkozari9271 I'm now convinced that Seagate HDDs are all PoS, a 2tb Seagate drive just crapped out on me a few days ago after using it for game storage for just a few months. Wasn't even my boot drive but I couldn't boot with it plugged in just like you.

[@Ren-bj4xm]

@@cloudyyy-._.-999 The Drive will show "unhealthy" in Disk Management first. The Disk will still operational but you'll notice some file will be corrupted, sometimes load slowly or sometimes the drive is missing in windows explorer until you restart your pc. Happen to me first and i'm using it as normal for 1 month procrastinating to backup the drive and then kapuff 2 TB data is gone. Luckly its my junk drive.

[@WickedRibbon]

I know this video is old, but I was totally gripped. There should be a reality TV show around tech industry catastrophes.

[@SonariNeiracchen]

WickedRibbon or like documentary stuff, those intense drama shows

[@TechnicalTatertot]

there totally should! I would watch every episode

[@zyrecha]

count me in!

[@freshjnew]

Now just go to a producer and pitch your idea. Because it's a great idea.

[@Aeikon]

Isn't that basically The I.T. Crowed? Also, The Office, if it was a tech business?

[@activatewindows7415]

"This is why it is a good idea to call tech support, because the second you call them, you will solve the problem on your own." This definitely applies to me.

[@SuperNGLP]

As someone in tech support i can tell you: you are not alone. I get at least one call a day from people who are like: nvm figured it out. bye.

[@jettburmester1613]

Or as soon as you walk into the IT Department at school your laptop works!

[@achtsekundenfurz7876]

@@jettburmester1613 That doesn't really count. You just entered the IT Aura, a.k.a. the FIF (Fix-It Field).

[@dragon67849]

@@achtsekundenfurz7876 Is this a "truer words did never dun spoke" thread?

[@nickpayne1056]

@@jettburmester1613 the fucking WHAT at school? Christ when did I get old

[@robertdascoli949]

"We'll have a better backup solution next time." Says everyone after their first backup recovery disaster.

[@Neatherback]

@@audigex wise words!

[@manuel0578]

how about ANY backup? RAID is NOT A BACKUP.

[@bennicholson7782]

@@audigex additionally, an untested backup is not to be treated as a backup

[@altmandma]

@@manuel0578 its bullshit I cant thumb up this comment more.

[@plumixtee3496]

this is basically the USSR excuse after Chernobyl happened

[@Lea-pe6iw]

It’s like the 5th time watching this now. Such good rewatchability value honestly. It’s like a mini-action movie

[@mtftGames]

Same here, I come back every so often

[@sebastianschon3141]

@@mtftGames same here :D

[@Marcos87600]

I've probably seen it like 20 times

[@jehare271]

Same

[@effeKtSVK]

Exactly my thoughts, just rewatched it as well

[@kilppa]

"NONE. OF. IT." Damn, Linus can be intense when he wants to. Sure the music helps, but this could be a really nerdy thriller movie.

[@shutout951]

Having dealt with even some minimal it work at work, i can tell you that line hit me like the balrog in moria.

[@Codex_0613]

@@shutout951 if they had lost all that data, that's potentially months of work, raw footages, WIP projects, ideas, scripts, etc... down the drain to say Linus were pissed off is an understatement

[@shutout951]

@@Codex_0613 oh absolutely. Like i said.

[@Plumbumismyname]

@@Codex_0613 months or even YEARS

[@Raphael-co5vq]

I love the concept of nerdy thriller movie

[@ChrisThe1]

I watch this about once a year to keep my backup practices sharp.

[@jdatlas4668]

Oh, same. Had a drive failure just this week and was reminded of this video.

[@BitcoinLifeToday]

That is so true! I learned from my mistakes!

[@TitaniumTurbine]

Same here and this is a backup modernization year.

[@Ben.N]

Same

[@isaacgober1128]

​@@milesfarberHOW

[@carnagefrenzy5080]

All time favorite LTT line: “Reality TV ain’t got nothing on IT work”😂😂

[@captaincoffeecake3595]

Meanwhile on Keeping up with the Katrashians - Kim can’t find her Prada toilet paper and thinks kanye took it

[@aleksanderokonek4952]

Whenever I feel stressed I watch this... I now feel more stressed.

[@onewhispergaming6435]

Today, Jessica's processer isn't working and Jean's server isn't working

[@TidalWavesLion]

He really isn't joking though. It's really that stressful

[@KaiEpic]

Its the truth...

[@haoss69]

you know shit goes down when linus calls for help

[@laxmitty21]

Very true

[@ambassadorofpain1]

+HaoSs Linus always calls for help, because he's incompetent.

[@ichihaifu]

+John Smith Not necessarily incompetent, but when people panic they tend to forget things. But I must agree that all of this comes from the fact that they didn't think far enough, which indicates inexperience.

[@ambassadorofpain1]

+ichihaifu When people have inexperience I generally wouldn't call them incompetent because that's largely unfair. I call Linus it due to the fact he has been told time and time again to perform tasks a certain way (by knowledgeable people) but never listens and ultimately fails. The networking and storage solutions have been a huge issue since moving to the new office and the biggest example of his incompetence, and in some ways negligent behaviour.

[@wileycsg]

+John Smith 2.1 million subscribers and he is incompetent? huh?

[@blackbox64]

"This is why you should call tech support, once you call then you'll find the solution yourself" Truer words have never been spoken.

[@meltedgyrofpv8477]

Always fixes my wifi

[@a-triangle2735]

Often, right at the moment, the tech answered your call. "How can we help you ?" "....uuuuhm ... nevermind ...."

[@a-triangle2735]

Mostly, because , while ur on hold, you start thinking about the question they are going to ask: - Is the cable connected ? - Is the power on ? - Did you reboot ? - Did you check the BIOS usb legacy devices ? You just go through that checklist in your head. Thank god for tech support phone numbers.

[@jakobole]

If I don't answer my phone at work and have to call back, 9 out of 10 times my colleague fixed it themselves.

[@seaston]

this should be the golden rule of IT

[@eccodreams]

6 and a half years later and this is still the best LTT video, in my opinion.

[@laytonfoster50]

Agreed

[@hipithautaa]

I watch this every time KZbin recommends it 😂

[@ghomerhust]

under extreme pressure, they worked through it to come to a solution. it's a job very well done

[@laytonfoster50]

​@@LaSombraa his name is Nick and if I recall correctly he works for IFHT films which is a mountain biking channel. They're pretty cool actually

[@waluigi356]

Some people watch soap operas or thriller movies. We watch IT work.

[@fransweijdener2093]

Waluigi why arent you in smash

[@roberthanson2636]

"IT work" isn't ever as exciting as this video, because real sysadmins wouldn't set up a monstrosity like this to begin with. It's guaranteed to break.

[@DavidAnderson-fr8ii]

This looks about like 2 of the companies I have worked for and No they never did learn. Plus they always blamed someone else for the problem.

[@GaryDrumm]

We may have a problem. 🤔😜

[@BixbyConsequence]

Some people juggle geese.

[@Shadow7758]

You know shits hut the fan when Linus has to call tech support

[@hybridram8129]

Jabba had a fan?

[@hybridram8129]

12 Volt? 110 Volt?

[@hybridram8129]

Oh Yeah... Yeah that too.

[@perwestermark8920]

Linus isn't a very deeply technical guy and he is to a large part emotions. Which means he doesn't think forward. He didn't read up on the effect of designing a server that stripes three RAID-5. He did not plan ahead for the consequences of hardware failures or malicious software. He likes things big and fast and lives for the moment. When things go wrong he is a trial-and-error guy. He has the charisma and personality to do well with a tech channel presenting hardware. But he's the wrong guy to design or administrate server hardware. And that's the reason he thought three RAID controllers and parity meant "safe" and skipped backup until it was obvious the machine had issues.

[@LFCFootball14]

yh hockey ultimate team the fan

[@hadleytrounson7110]

Hats off to Linus, CEO and front line IT support. Would've been easy to bark at staff to make this their problem, but instead he showed leadership and saw the problem through himself. 10/10 would watch again.

[@kennethschultz6465]

I wonder why he newer unplug PSU or use Antistatic 3m wristband Or any What so ever prokortion against failiurs .. Never made a PC or server with out my earth strap

[@bln35]

@@kennethschultz6465 he's said before he often puts it on an ankle for serious builds in the past

[@Sekaisien]

@@kennethschultz6465 You yourself said factories (professional environments) strap their cords on the ankle. Why wouldn't Linus?

[@kennethschultz6465]

@@Sekaisien why woulden:t Linus !!! Doe's Linus moove!!! Walk around ???? You know ther is a earth lead right !!! So No he is not theatherd at the ankel!!! And What would IT show and learn All the wiever's !!! If IT was on the ankel and not on the wrist !!! The tabel ain't Antistatic either !!!! So why use ankel strap and not earthed tabe???? This is NOT ANTISTATIC ENVIORMENT AT ALL!!!!

[@seefzi]

@@kennethschultz6465 I have never seen the English language used in such a unique way.

[@michaelbretall8812]

"NONE. of. it." Damn, there was some genuine venom in that sentence 👀

[@KamenRiderRaiden]

Thats the tone of voice that precedes someone getting beaten with that server Linus was working on if they said one more thing

[@techhelpportalextras3007]

Nunavut?

[@2LucasKane3]

7:48 kzbin.info/www/bejne/bWWlq4aKhqiflcU

[@OldManBOMBIN]

I've been there, lmao "This is the case." "This is?" "Yup, this is how it is." "It is? "Yeah." "Is it some other way?" "Nope, this is how it is." "Are you-" "Do not be afraid of the inevitable, friend. Life has led us to this point, and now we must play the hand we've been dealt. There is nothing that we can do but move forward."

[@marcusthegamer348]

@@OldManBOMBIN This sounds fake as hell but sure bud

[@MC3pk]

With this video WeRecoverData just received huge street cred

[@RyanRunz]

Yup, nice one you guys.

[@kdojeteri]

+Señor Tomas For me they lost their cred when I heard this: 16:30

[@Steamrick]

+Josef Hornych I'm not an expert... are you sure that the drive letter changes were not due to Linus switching the raid cards around in his effort to get something working?

[@kdojeteri]

Linux does not have 'drive letters'. Only mounting points. Saying 'drive letters' in Linux doesn't mean anything.

[@gentuxable]

+Josef Hornych One way you talk without end with no reason or you use terms the user may understand. How many times I had to explain but it didn't reach! It's just experience to use wrong terms to be understood, I have to do it regularly just to have a short sentence and not to explain everything. Especially when users are worried they want to know EVERY detail.

[@x105w5]

7:40 Linus: "none of it" Cameraman: "some of it" Linus in CEO voice:"NONE OF IT"

[@alejandrozamora3468]

I noticed that too. Gave me a nice laugh.

[@sacfsd23]

I imagined Gary Oldman ala Leon the Professional "EVERYONE!!!!"

[@Lambda_Ovine]

We don't get to see Linus on CEO mode very often.

[@voidvev]

ya lol, but i guess i shouldn't laugh he looked ready to do his 3 Linus steps to freak out curse freak out and then cool down with some resolution

[@brycematthew3115]

@@voidvev It's a really hard moment because the other guy doesn't understand why it wouldn't work, but he keeps asking and therefore making linus repeat that its fucked.

[@cyberaxe_lznj]

With so many comments I doubt you'll go back and read this but... I'm a technologist in a small town in Colorado. For the last 20 some years I've been a lonewolf and a few times been stuck in a server room for days after update crashes or corruption, it was very isolating and stressing. Seeing you deal with your server room was comforting and therapeutic, I've since watch tons of your videos and love the channel. Keep up what you do, it's very stressful and you're doing a great job. You rock!

[@IoriTatsuguchi]

I've always been accompanied by someone else in the server room all the time so that's beyond my imagination. Cursing or sobbing in a room with bunch of iron boxes must make me feel like shit..

[@andrewyork3869]

@@IoriTatsuguchi that gives me a migraine, he is clearly someone who knows what he is doing. Someone else didn't listen and a train wreck that results in loss of years of life followed. Granted a perfect storm could have occurred, but I find the latter more likely.

[@llaith2]

I once witnessed someone destroy the company we were in by making a mistake restoring a HP Raid array on the Sunday. I was there for another reason (working back on some software), and he came in because of a HDD failure, we talked a bit, he told me he was stressed because the system was tricky, then boom, he screwed it up anyway. I don't think it was his fault, the system *was* extremely confusing. When he realised, his face went white. He nearly broke down in tears, and he left. The next day the news was announced, 100% data loss. No backups because they thought the raid array was a backup. Ho ho. That's when I learned it wasn't. Luckily there was some files on many staff desktops (this was the late 90's, no laptops in common usage yet) and they company limped on for a while. I moved on to greener pastures, and he quite and became a pilot. I've never used RAID since.

[@andrewyork3869]

@@llaith2 raid when properly applied is a good safety net, how ever you hit the nail on the head, it's never a back up.

[@cyberaxe_lznj]

​@YOURALLSHEEP Thanks for sharing this constructive and informative comment. I'll take it to heart and think on it for a while.

[@markgergel1717]

5 years now this is still my favorite video. Working in a data center the problems are real. People are waiting to work and you are left talking to someone over the phone praying they can help you with your issue. I think the only thing missing is since Linus is the owner he is saved from upper management calling down every 5 minutes for a status and "Do you know how much money we are loosing by minute?" speeches. (Also a lot of told you so's from us about converged and redundant infrastructure) :)

[@marios3202]

Dude, this video has such good re-watchability. This is like the third time I've watched this..

[@bnmc]

4ourth time for me.

[@drakoclash5715]

Max Poma 1rst time..but Omg wanna watch this thing Consecutively

[@nurhaida1983]

Mario S same here. linus is really awesome

[@robsmors]

I was just about to say the exact same thing! It`s like a LMG mini drama film!

[@Dr.RobertTHC]

Mario S SAME. I've gone through a situation like Linus had before, much better when it's not happening to you.

[@davidbluecame]

This video is full of feelings of terror, loss, regret and guilt (for not having setup a proper backup system beforehand). I can feel your pain and I'm very glad that you got everything back. I think this video is extremely useful to teach other people the value of backups and to show a way to potentially recover your data. Well done and thank you for this! I did not find it "funny" as you said but I was gripped by it from beginning to end.

[@d.romero3014]

The lesson "do backup constantly" was taught by Microsoft to everyone with the last Windows 10 major update recently.

[@DaWalkDude]

"...A thousand dollars in rental equipment" Back then, not even Linus himself knew he would own that thousand dollar equipment himself

[@Something_Disgusting]

Recommended feed?

[@DaWalkDude]

Something Disgusting took a trip down memory lane after that 3080, 8k video

[@Something_Disgusting]

@@DaWalkDude You know da-real shitz.

[@Burger19985]

and drop 1000$ cpu's without batting an eye

[@jamesdong1792]

Whenever I feel stressed I watch this... I now feel more stressed.

[@zoey4284]

0:25 Taran: I'm actually doing some writing right now. **Feet up on desk reading a book**

[@thegermanhero204]

I think, he was searching Inspiration

[@pavelnovak1350]

When you’re moving files, you don’t need to do anything :D

[@jaiden051]

Writing words in his head

[@ahoj113]

wreading

[@ShippoHsu]

I think it's writing some data to the server

[@ahears1]

If your data doesn't exist in two seperate locations... it doesn't exist.

[@avrona]

Not sure that's how it works bud.

[@nathanh775]

@@avrona you must be fun at parties

[@OctoomyYTOfficial]

Ah yes, I should listen to the creepy pasta that scared me as a little wee lad.

[@StanbyMode]

@@avrona facepalm 1000

[@sunnysanjo]

@@avrona r/woosh

[@JmanEspresso]

Idc its four years ago, Im just realizing it now how awesome Linus is in this video. We see Linus doing what linus does, his usual thing. But look at the context. Put it in your own terms. You work at said company. Crisis happens. Does the CEO drop his workload and fix the problem ENTIRELY himself? While keeping the staff calm and reassured that things will return to normal? Or the the CEO blame lower management and dump responsibility on them? More often than not, its the latter. Any of Linus' fans who have been around since the NCIX days probably became geeks because of Linus. I built my first rig in 2009/2010, and was inspired that I could handle it thanks to Linus. Pretty much any of us always would say, working at LTT would be the dream job. Or at least a really cool job. But now its obvious it would be a great job that is personally fulfilling. WOrking for a guy like that, you know he cares about everyone the same. From his partner in crime, to the lowest end employee, the equivalent to the guy who wrangles the shopping carts, hes got your back the same as his main go-to guy, whoever that truly is.

[@jesse7770]

Very well said!

[@adame7277]

So true. Most higher management don't react like Linus and take hold of the ship. Instead they cower and blame everyone below them. Love this guy. Honestly he seems like a good guy to work for and treats his ppl well

[@lovebirdmic]

I feel like I'd want him as a friend... Like, a real freind xD

[@calibcrockett]

Nice try, Linus!

[@henrydorsett6076]

@@adame7277 Well, CEO is just three letters tbh. Some CEOs would not, but then they might have 3 meaningless PhDs in Economics and 300.000 employees around the world. (My daily ^^). A challenge to your perspectives here ladies n gents. :)

[@MustafaMuhalab]

19:26 "We should physically print all of our videos, every frame on the video, on a paper!! " LOL

[@brycejohnson7]

Mustafa Muhalab

[@wright96d]

You win.

[@victornpb]

stone is much more reliable than paper! I guess until you drop it... so one must have a flexible alternative as a backup as well. like wood

[@hydrochloricacid2146]

The real way to save videos would be to print the data on 5 slot punch cards.

[@wweislife5685]

No the real way is to transcribe every pixel onto a single moniter for every frame

[@1lomi901]

It was certainly one of the best swegway for werecoverdata ever. This is how it's done. Do your job properly for the right person and you'll be rewarded at some point

[@skinnykid7783]

He handled this like the perfect boss Doing it himself and keeping people calm

[@ash.mystic]

The difference between a boss and leader!

[@GregorioStyreco]

He wasnt calm broski. When he said "NONE OF IT." You could see how he had to cooldown for camera reasons. He wanted to explode (he did inside), but he didnt. THAT what makes a good LEADER.... and 5000 dollar intel sponsorships.

[@CatTerrist]

Bullshit. He edited out the temper tantrums

[@MMMHOTCHEEZE]

@@GregorioStyreco AMD*

[@pringlelays159]

Did u try turning ur server on and off again?

[@Sora-o]

/facepalm

[@MuhdIqbal]

lol

[@mlgbanna253]

RLLY?

[@austin1579]

+Pringle Lays Your joking right? Right? Please say Right!

[@ramenns9693]

+Pringle Lays It's a hardware, you can't fix it just by turning it off and back on.

[@aeons5917]

I know what the problem is... You never added RGB to the server

[@thehydragon6368]

Filthy Hanzo Main.

[@mianali5680]

Filthy Hanzo Main nahhh cuz u're a Hanzo main. stop playing Hanzo and throwing games in comp :p

[@stefan23006]

Nah im genji main LoL

[@mianali5680]

ohh gawd shimada weebs destroying others players competitive rank :/

[@yo_its_gingey5329]

hay he had some blue neons if that counts

[@m0o0oeh]

It amazes me how much this video still enthrals and grips me despite the fact I’ve watched it dozens of times. Another desperate issue that requires dozens of man-hours to resolve but has a happy ending.

[@TyranitarTube]

hilarious to see people calling this a clickbait title. Maybe "We got it all back" will make the 23 minutes of suspense more enjoyable. And for sure, Reality TV aint got nothing on this. This is as real as it gets

[@KamenRiderRaiden]

Well i mean, for one, theres no multiple takes to make sure they get the "spiciest" angle for some of the drama, so that adds to the authenticity

[@mi-yo8401]

Linus: has whole server room. Me: has two usb sticks.

[@chrxsfn]

Bitch, please I have an U L T R A 1 T B U S B T O T A L L Y N O T F A KE

[@miko8732]

I use my 5th gen iPod classic with iflash as external SSD

[@Ethodica]

i have one come at me

[@Yes-----]

I have 6 32gb SD cards connected to an adapter

[@thomaspiekos3440]

@@Yes----- I 48tb hdd + 2tb ssd Both died last week 😭

[@andrejwalilko634]

Lesson 1: raid is not backups. Always have backups. Lesson 2: never use raid 0 on anything that matters; it just increases the chance of failure. Use raid 10 or 60. Lesson 3: on-site restore techs save a lot of time and headache :) Lesson 4: number your drives and cables!

[@ChrisWijtmans]

RAID5. not raid0

[@andrejwalilko634]

they seem to be using raid 50, multiple raid 5s under a raid 0. still better than plain old raid 0, but loss of two drives in one raid 5 element is deadly, so you still technically only have the ability to lose one drive before you need to panic. secondary to lesson 1 :)

[@ChrisWijtmans]

@@andrejwalilko634 i would use raid for redundancy primarily and then speed. and still have off-site backup hehe.

[@MrAntarpreets]

@@andrejwalilko634 Why would you ever choose to do multiple raid 5s under a raid 0, wouldn't the opposite - multiple raid 0s over one raid 5 a much better option, or would it turn out to be slower for some reason. EDIT: I guess It all comes down to hardware support.

[@andrejwalilko634]

@@MrAntarpreets the raid 0 on the outer layer gets you the speed of striping across multiple volumes, and the raid 5 on the inner layer provides the cost efficient redundancy. Raid 0 on the inner layer and 5 on the outer is not a standard organization. It would probably be slower, as there is only one checksum mechanism. Multiple checksum points on multiple raid5s lets you calculate more at once, so it should be faster.

[@jamesCondon6]

This brings me back to a quote I heard somewhere before: There are two kinds of people in this world. People who take backups, and people who’ve had a disk failure.

[@RsKnDR0991]

Nice one!

[@smellymomo]

With some more music, documentary-type interviews, some sci fi effects and a deep-voiced narration in background would have turn this into a Netflix blockbuster.

[@calebfuller4713]

Except instead of Linus Sebastian, its Linus Torvalds, and the server was one holding the new version of Linux he was supposed to give to IBM for their new Z series mainframe...

[@heitorfreitasf]

nn

[@heitorfreitasf]

nnn

[@heitorfreitasf]

nn

[@heitorfreitasf]

nn

[@Aaron-px2hr]

Linus calling tech support, that's a first

[@pdubowner]

Probably not.

[@rly907]

R/wooosh

[@hunterblechner4948]

Lol!!!!! I will at a school and the some of the kids call tech support. Lol

[@Bonta22]

@Money-Gucci-Bitches same here my friend. i am actually on hiatus from working since system administration work fucked my brain really really hard.

[@DaneRamsourSr]

@Money-Gucci-Bitches D-.........for grammar and spelling.

[@thegettokidZz]

This felt like a reality tech tv-show episode. At the end I was expecting to hear a narrator say "Next time on Linus's servers, see how linus and the team tackles the problem of ..."

[@em5345-]

Accidentally releasing a development photo of the server

[@timthomas9105]

This is probably the MOST REAL, Behind the Scenes, video you've ever done. As a technician myself, there's a place you go in your head, a Focus that lets you SEE the process, commands, or schematics of what you are trying to accomplish. Having your team of employees express a level of confidence in your ability to get it done really shows through. I'm a relatively new Subscriber to your channel, and happy that you are out there making these computer related videos. Great work. TT

[@herumuharman6305]

I have a bad experience with server data loss. One day we learned that our surge protector was busted, we learned it the hard way. It was a rainy day when a lightning strikes a nearby power transformer. The electric surge fried one of our servers. The server won't boot up, and when we hooked up the HDDs on another system, it won't recognize it. We were a hospital, so that's mean patients record and billing, that was our most important data, and no, we don't have an off-site backup. I have to bring all the physical drive to a nearby data recovery techs, which is two islands over (this is Indonesia, we don't have that many good data recovery companies). Luckily it was Friday night, and the tech agree to work on Sunday (God bless his soul). So I board the airplane on Saturday morning, drop the HDDs in the afternoon, and spent the rest of the day in my hotel room. I'm too nervous and stressed out to go out and look around town. I got a call on Sunday afternoon to come in and take my HDDs. They assure me that they've done everything that they can, and they were sure they got at least 90% of our data back. So I pack all my HDDs, my new HDD that contain all our rebuild data, and get on the first plane home. After we hook up our new HDDs into our new system (my co-worker set it up while I'm away) we check it to find which of our data is missing. And it turned out, none of our data is missing. The tech manages to recover all of it. That was the day when I put data recovery techs on top of my IT worker hierarchy.

[@sahibjot01]

Heru Muharman fun to read

[@MrSnowman777]

Heru Muharman interesting

[@rohanofgondor]

"surge protector was busted" it sucks af when the one thing we buy to protect us from that kind of situations, get damaged and mess our main component.s

[@deGoodMe]

gan, boleh tau nama jasa apa dan dimana? buat refrensi saya kalau tiba2 mengalami nasib yg sama. terimakasih.

[@fadie.b.5120]

Omg wtf i would die from that stress ..... Lost data from a hospital this doesnt sound good

[@hokeypokeyy8551]

19:34 "Yaaay for hiring other ppl to do our jobs" literally the most true words ever.

[@qbflicks]

Hokeypokeyy ii

[@AaronBowersable]

outsourcing

[@phoenixfront]

Whenever I feel stressed I watch this... *I now feel more stressed.*

[@phoenixfront]

First reply 😯🤣

[@michalthemichal3550]

@@phoenixfront Obama giving Obama a medal be like

[@walidfakhfakh3660]

@@michalthemichal3550 mccluer High School ma to do about 10 years ago video

[@phoenixfront]

@@walidfakhfakh3660 😂

[@theslader]

as a IT engineer myself, I know this roller coaster to well. I have had the pleasure of riding it more times than I would like to. So I still look back on this vid from time to time, just to keep me sharp knowing this ride could happen without a moments notice.

[@HenrichAchberger]

at 19:27 "We should physically print all of our videos frame by frame" Denis is awesome

[@MaximaToluene]

+Henrich Winside Achberger yeah but denis is awesome

[@kimo43334wrt2r243t]

+Henrich Winside Achberger Nick in a nutshell

[@SakariNy]

+Henrich Winside Achberger Maybe they should use PaperBak for backups. It lets you print (and then later scan!) your files. The papers look kind of like QR code and you can apparently store 0,5MB of data per paper :D

[@wanraiqal]

+SakariNy then you would need approx. 46,080,000 sheets of paper to backup whonnock server. 960x24x1000/0.5

[@eIucidate]

+SakariNy Cool!

[@DarkSansTV]

I can't seem to stop coming back to this video. This video was so good.

[@cyberfox9713]

Same

[@pcpatch9243]

So true

[@xplinux22]

Same! It's well edited and the carefully-placed tense music really kicks up the atmosphere and suspense at all the right times.

[@DarkSansTV]

@@xplinux22 Agreed. ;)

[@ztippie]

This is my comfort youtube video. I watch at least twice a year.

[@MathewZ788]

Linus's emotions matched mine when my first pc didn't boot up.

[@FCHenchy]

I went through exactly that yesterday! Stayed up till 02:00 to figure out I had used the wrong outputs in the power supply.

[@pablo2001nov]

I had it when my phone's 64GB sd card failed.. Like I literally have everything in it with only 28gb of it Backed up. Thankfully , only my exfat table was corrupted , so I could recover it all ( took whole night) and the only thing I lost was 2 movies I downloaded on the day it corrupted ( which is nothing )..

[@SaurusPlay]

hahah, it happened to me before, I rebuilt the whole thing to figure out that the power supply was off

[@dev_hazelify]

reminds me of booting my first ever PC. psu switch (on/off) was flipped off thus not allowing the power button to turn the machine on. spent a solid 5 minutes in a frenzy of frozen panic before i checked the back just in case and found out the switch was off. flipped it to on, booted the thing and it started up. one of the biggest reliefs in my life to this date

[@missingno2401]

reminds me of when i fucked up my dads computer with viruses and staying up until like 3 am or so to get the computer working again. i was 5.

[@KeppyKep]

I remember watching this video years ago and it almost entirely going over my head - I couldn't follow it at all. Today figured I'd watch it again, cause it's been the 'recommended video' at the end of a few recent LTT videos, and I'm happy to say I easily followed the whole thing. Yay for learning!

[@MeLikeBigBoom]

3:01 what goes through the minds of the folks in charge of packaging will always confuse me. "Hey Jim this area over here seems a little bland, mind adding something there?" "Way ahead of ya Tim, I already added this completely unrelated face of an unattractive woman there."

[@QuackZack]

+MeLikeBigBoom I sorta think they didn't even try to hire a graphic/design artist. But hey, what's more abstract and eye-popping than an oily western woman on the cover?

[@kaeden5590]

+MeLikeBigBoom hey im one of your biggest fans

[@0range1968UK]

+Kaeden Wilson a 250mm [super?] bigboy from Antec? now thats a big fan !

[@ITheWalkable]

Based on your comment i take it your a funny guy! Ill give your channel a shot! Thanks for giving me a laugh

[@VithorCasteloTutoriais]

+MeLikeBigBoom fun comment, but we need more videos about the upcoming unturned 4

[@caparzzo]

this episode is more intense than the star wars movie

[@Zyphera]

True Story.

[@iMarcism]

+caparzzo Yeah Like how you see Jaba the hut joining the dark side and being the new sith lord

[@vernerid4766]

Linus dies at the end of Pirates of The rings

[@powerhouse884]

I was expecting a couple of months before people who loved Star Wars started talking trash about it. Cuz thats what teenagers seem to do these days. Now Avatar is trash after it broke billions and billions on records. People from this generation are really sad. Btw if that was not your intention then i apologize, it just brought me an issue very common lately.

[@atl6s]

+caparzzo snape kills dumbledore

[@dashw]

"pcieport...errors" *tense music kicks in* me : I DONT UNDERSTAND ANYTHING BUT THAT LOOKS VERY BAD IM SCARED

[@Kid574]

That last line is pretty much my parents thoughts every time i boot up my Linux machines 🤣

[@captainzero119]

wait why is there boss music? "NONE OF IT"

[@Nunya58294]

That's from the kernel. I've not ever seen it spit errors continuously like that. Normally it kernel panics and the system halts

[@isaacbenson1631]

@@Nunya58294 on systems with failsafes you see this kind of stuff usually I know kali and tails I’ve seen this happen a lot because if it shuts down by anything other then the shut down button it deletes itself if you configure it to do so as per my understanding however I’ve never seen this happen with other distros and use cases though it’s kinda weird maybe it’s something to do with its own failsafes in server designed software

[@sharkiplay6664]

I was the thousand like

[@tariqbojic7632]

5 years later and people are still coming back to this video, I've watched this video 5 times and it still manages to build suspence even though I know the ending!

[@oops8685]

"We should physically print every frame onto paper" Classic Berkel

[@Ares-5933]

AresPhobos nice name

[@tonychen9499]

AresPhobos 500th like

[@angryidahobusdriver]

AresPhobos I'd be cool honestly.

[@acorndrumneptune]

i miss him now

[@SuperCookieGaming_]

imagine a breeze mixing up the papers then having to reorganizing it

[@Myshkal]

7:98 That “None of it” was said with so much genuine anger and frustration

[@mazzalnx]

I don't think that's how seconds work

[@MinePlayersPE]

@@mazzalnx its in metric

[@H12-q7x]

@@mazzalnx at least KZbin's engineers where smart enough not to trust user input.

[@H12-q7x]

7:48 r/therewasanattempt

[@macoud12]

Because RAID0(RAID5(8 drives),RAID5(8 drives),RAID5(8 drives)) is quite risky, if one RAID5 fails, all the data is gone.

[@alexandrefossatifilho558]

I'm for sure a computer person, I almost cried with that recovery. Imagine a whole stressful week in your job and you finally got your data back. This is so satisfying.......

[@kalanireece9289]

Here in 2023 looking back and realizing this was my true calling. Here 7 years later as an IT Technician Level I and on-site technician working on business critical servers for multiple clients. Thanks LTT for showing that we, as IT specialist, can hire other people to do their jobs ;) Thanks guys for giving me crucial knowledge that I didnt know I would actually use!

[@Corristo89]

7:49 "None of it." Linus looked like he was about to crack the guy's head open just for breathing in the same building. This is why you always follow the 3-2-1 backup rule with critical data (3 copies, 2 on different devices and 1 off-site (cloud, rented server racks, co-location, etc.).

[@KamenRiderRaiden]

I said in a different comment thread, thats the tone that precedes Linus grabbing that server case (yes, drives included cause he thought they were all dead) and beating Taren severely if he had said one more thing after the third reiteration of " *_NONE_*_ of it_ "

[@jdatlas4668]

@@KamenRiderRaiden and I wouldn't blame him. In a striped config losing any individual group means you're basically screwed, and he was pretty clear on the stuff they don't have backed up. This is a stressful situation, and if I was in his place I wouldn't be charitable towards people asking unnecessary questions, even if they seem reasonable from their perspective.

[@raythepizzaguy9721]

Did you take the cartridge out and blow into it? That usually works for me.

[@TrupimOkiemTV]

now THAT was funny. this video is some weird mix between hardware porn and hardware horror, butr funny? not at ALL. :D

[@Peng_Pong]

No dude, you're supposed to turn it on and off!

[@kevins5600]

Did you set it to wumbo?

[@synexiasaturnds727yearsago7]

Try doing "POKE 59458, 62"

[@kernel_data_inpage_error]

7:44 one of those moments you need people to remain silent so you don't kill anyone

[@AppallingScholar]

Underrated comment of this video

[@omnacky]

If I was the cameraman I would have made some joke about Nunavut

[@nicky2575]

@@Alexander-od6fi Jesus Christ man calm down

[@quixfilmz3409]

@@Alexander-od6fi nah it's content. If they all remained silent then the video would be boring as

[@Amy_A.]

@@Alexander-od6fi You have to realize that yes, he's interrupting and yes, it absolutely isn't helping, but they're making a video. The cameraman was trying to clarify, both for himself and the viewers. You're acting like he's shoving his face into the situation without permission but do you really think Linus would let him stay and record if he was actually causing harm? Linus is the boss, he's allowing him in specifically to document the recovery process so they can make a video of it later. I do tech support and coding so I know how stressful it can be to be constantly interrupted, but there's no need to be so negative about a video totally unrelated to you.

[@JessicaFEREM]

I know these videos are harder to produce, but I love these behind the scenes vlogs, this is unironically better than any reality TV show

[@DIYPerks]

Edit: apparently this comment was a 'spoiler', so I've erased it. Seriously though guys, if you don't want to see the outcome of a video before you watch it then don't read the comments :P

[@moritz5282]

it's youuuu

[@AleksandreMzhavia]

+DIY Perks Ah man, you spoiled it. :D

[@TheTransfusioNetwork]

+DIY Perks Awesome! Love your channel man!

[@ihatemadmoe]

+DIY Perks spoiler alert

[@jakehowell9394]

+DIY Perks OMG I Love your Chanel and I'm a massive fan! you make such cool stuff and have inspired me to create my own contraptions too!!! Also I completely agree, you would've thought that Linus would've backed everything up considering he has quite a few videos where all he talks about is how important backing up your data is, it's kinda ironic but I'm glad everything's okay now :) ps. Thanks for making great videos!

[@CapyBrotha]

I felt like I was watching a scary movie...

[@-hobbit-9481]

So did I!!

[@TheMoose2010]

+Adronomics This is scarier than any scary movie I've ever watched. Mainly because I've gone through something similar before, but with less important data.

[@aminathleem7989]

+Adronomics this is way more intense!

[@TheJohn8765]

+TheMoose2010 Yup. After multiple HD failures, I've got a networked nightly cloud backup for anything personally important and most of my large files are on my NAS. Nowadays, it's cheap to run a backup HD off your router with cloud storage for your important files.

[@adrianbornabasic7499]

+Adronomics Me too, it was very scary!

[@JerryRigEverything]

How much did that cost you to get restored?

[@assettobam4880]

That was probally 5-50k

[@deadalpeca8099]

Holy moly is this Jerry with such low likes?!

[@arasakaa]

@@deadalpeca8099 That was 5 years ago

[@splasherfinek]

That was 5 in scale of hardness

[@icecreamjunkie6790]

The true cost was getting a better backup solution

[@seantellsit1431]

I'd like a follow up to this talking about A) why a raid 50 was used with 3 stripes. B) Why failed. C) and What you've moved on to D) why is the new platform better for peace of mind)

That raid/disk configuration really boggles my mind. Why would you configure it like that, it’s like creating raid and then creating multiple single point of failures for all data.

[@jdatlas4668]

Educated guesses on a) and b) are "each card has limited bandwidth and striping together seemed like the easiest way" and "something was wrong with the motherboard and it messed up one of the RAID cards", respectively. I assume c and d are actually covered in the newer videos about NVMe-based Whonnock and their new backup scheme respectively.

[@mycelia_ow]

@ it's Linus, what else would you expect lmao

@@jdatlas4668 If you do setup like that, i'd say it's more on the lines "i don't understand what i'm doing" or "let's gamble a bit".

[@jdatlas4668]

@ I mean, I’m not going to dispute that. But especially in small businesses this sort of thing happens a lot, and I’ve seen far worse.

[@lastwymsi]

You know your a tech head when this was more dramatic and thrilling than normal Television.

[@hamsterruler]

Wymsic_ ...

[@lightgamez4561]

This comment is so true!

[@joeprestera2239]

Ya and triggered my PTSD from 25y of IT.... gone through this before.

[@NebRetalsJr]

Joe Prestera lol so funny

[@cant_kill_me]

for some strange fucking reason I want to feel a tension like that because although I probably had similar amounts of stress for different things, I for some reason want that tension and especially the relieve after it.

[@SimpleGunner]

i counted about 4 times my heart sank with his. who ever says IT is not intense has never experienced IT.

[@trilight3597]

+shawn zylla or worked for IT, I'm with you there.

[@GrumpyIan]

+shawn zylla I shadow the IT guy at the college I go to. There was a power surge one day and crashed the servers. We had to put a new power supply in one of them and the other had a messed up boot sequence. It took us about an hour to figure out/ fix the error.

[@mitcHELLOworld]

+Dada blahblahblah

[@GrumpyIan]

***** you still repair compoopers

[@DeltaFish11]

+shawn zylla Well it really depends on what kind of IT service you are doing. If it's the basic "My computer is slow" that's nothing. If it's network or server admin services oh boy you are in a world of hell.

[@freddiegardner4776]

>Whonnock server broke >Understandable have a nice day

[@brycejohnson7]

HummusLord1337 oof

[@swagmaster-vn2dx]

anyone called me?

[@taylorjade136]

Can I get a * B O N E L E S S C L A P*

[@evennot]

understandable, not quite

[@bishopanderson5864]

Great day broke too

[@micahrufsvold]

The LTT team's production quality improves all the time, but the music in this one is a true master piece.

[@redyau_]

This is accidentally one of your best videos. I love watching the behind the scenes stuff way more than the actual content :D

[@geennaam1201]

i realy expected linus half way into the video to say: stay tuned for part two of all our data is gone in the future!

[@Reese_Asher]

+geen naam SAME HERE

[@valkyrie9561]

+geen naam ooh je hebt wel een naam! en ik zal er achter komen ook!

[@geennaam1201]

kwallen hoeven geen naam te hebben zelfs geen donkerpaarse kwallen

[@valkyrie9561]

geen naam maar jij bent een gewone kwal in ik vid dat ze we namen nodig hebben!

[@geennaam1201]

Fabian Miserus nouja mijn naam komt van een aflevering van spongebob van vroeger die ik mij nog half herriner. spongebob vangt een speciale paarse kwal en wil hem een naam geven, kan niet op een naam komen dus noemt hem geen naam en vond het bullshit dat je van google je naam moest invullen

[@franciscoaloureiro]

The moment that Linus hit enter, my screen went black. And then I realized that I was updating my Graphics card drivers.

[@dhiaiqbal]

Francisco Amaro Loureiro he Sounds Like A Cop In NFS Most Wanted '05

[@0x00Fyou]

did you have a mini heart attack when you saw the black screen?

[@rubberboa]

lmfao lol

[@blackgenesisishere]

yep ive gotten scared as i forget im updating nvidia drivers...lol.

[@kuhafiq4928]

Linus employees must be so glad to have a leader like him.

[@deadcat6085]

reality TV for computer nerds

[@touchofthorn1841]

The King Slayer lol yeah

[@heryccortes]

Cool N boy Xbox Gamer I understand like 3% and I still like it

[@fgv3357]

The King Slayer yes! this is amazing

[@HaydenX]

That's because it isn't a "dramatic recreation"...it is actual reality.

[@serenity748]

Fuck You

[@SommetiderHvorforDetRoligRolig]

Best horror movie on youtube.

[@ahmedimraan]

Haha

[@TheGoIdCat]

how is this a horror movie?

[@celestial3545]

The Gold Cat How is this not?

[@raptorjesus3894]

Did you try turning it off and on again?

[@MultiSilver3000]

+Miro Sturno lmao

[@tyreethomas6246]

+Miro Sturno like consoles press the power button to turn it off

[@invalidsudo]

not instructions clear, stuck into a dick fan

[@CoderMonkeyNathan]

+Jayden Holzhauser IT crowd flashbacks.

[@Exphiser]

+Jayden Holzhauser Well, it looks like it's turning itself off and on again...

[@Solothedrunk]

I know that "NONE OF IT..." look. I felt Linus so hard there; I felt the pain of this PHYSICALLY!

[@RenegadeFury]

It's pretty impressive how you guys manage to consistently fuck up everything.

[@JashtonnFS]

You guys are idiots. I think there's more value in the knowledge gained from events like these than a systems administrator who would be paid a decent salary to manage such a minuet infrastructure, in the scheme of things. Considering actual outages like this will be so infrequent, as well as the fact that the client size is so limited, a full-time guy isn't needed. Insulting people for being creative and intuitive and thinking outside of the box with regards to finding solutions to problems makes me occasionally wonder what's wrong with the world. Remember, this wasn't their fault.

[@xkmto]

+Terrapin GD the fact that he can do that and get the system to work is amazing

[@thefuckklay]

what would a "real" server administrator do differently?

[@JashtonnFS]

+TROLLFACE KNIGHT Exactly. That's the funny part. He'd essentially diagnose the same problem which wouldn't have been his fault in the first place (it wasn't LTT's fault) and would've undergone a similar data-recovery procedure.

[@JashtonnFS]

+D-MMA Quite right. Everyone learns from things like this, so if anything i'm grateful to LTT for uploading this. So many people, even the most elite of sys admins experience issues like these. Seeing people actually experience and have to deal with it is so valuable, and we can all gain so much knowledge and understanding from it.

[@ScrattleGG]

Oh my. I lost all of my data, school work, notes, family photos, all my youtube footage and videos, editing presets... basically everything I've ever done digitally was there, including most of my physical stuff I scanned in. Huge props to Seagate for helping me out getting my data back. I was fuuuuuuuuucked.

[@triton62674]

+ScrattleGG ATH Mx50s

[@Shangori]

+ScrattleGG And that's why, even for personal use, I have online data storage with backup. Dirt cheap and everything important won't vanish when something stupid happens here.

[@Postalpacifist]

+Jack Burton Parity certainly didn't help linus. It's not enough without multiple dedicated backups.

[@MaDmanEXE]

Why the F You were using RAID5? RAID6, RAIDZ2/3 and RAID10 are ONLY RAID's to protect such sensitive data

[@nijjijjion204]

+Shangori Problem is some universities don't allow schoolwork to be put onto online data storage sites. I'm guessing as being data for schools is private/confidential and owned by the school, they are worried about people/governments can access these files on the online storage sites and they won't allow it for that reason.

[@HazmatGamer9842]

Alternative title: Linus needs tech tips

[@typical_watcher4599]

1st Tip: First make shure you have back-up, or better yet backup of backup of your important files in the server (or at least, an incremental back-up)

[@alright7131]

@@typical_watcher4599 is it good practice to backup data on cloud?

[@typical_watcher4599]

@@alright7131 I think so, security experts recommended at least 2 backups, one in cloud and one offline. But of course, LTT have huge data and uploading always to the cloud for backup will always be pain in the end of large intestine.

[@kushiva7651]

This film is more dramatic and nervekeeping than 90% of hollywood movies - even 3 years later xd

[@haryholland]

Ikr when the harddrives wernt showing up it was super tense

[@samuraiartguy]

WORD. I'm still sweating.

[@nikolongar]

👹

[@pennyhu4795]

Exactly 😂

[@MrKoshV]

The music they use for each scene is very well placed and mood setting.

[@ryantoomey611]

This is why it is important to always backup your data.

[@mrmoomoo1016]

IKR!!

[@mika2666]

20 TB of data is a lot though, and they have double back ups :)

[@ken_sparrow]

They just moved. That was his backup.

[@dfortaeGameReviews]

Yeah, that's REALLY bad that they didn't have at least 2 backups. Different types and locations.

[@Happy_Free_Time]

You can do unlimited incremental backups to the cloud cheaply of all your data with unlimited history for about $60 a month. If you start early then 20Tb isn't that much, and with a Gbit connection it should take max two days to recover all of that. We normally just ask our provider to give us a daily bandwidth boost when we have to restore backups.

[@ewenchan1239]

This was incredibly suspenseful and gripping. Reality TV has NOTHING on this. As someone who has experienced data loss - I think that this is something that a LOT of people can relate to, especially given the kind of volume of data you're talking about. *yikes* You guys DESPERATELY need LTO tape backup. DESPERATELY!

[@ewenchan1239]

@Asdew Actually, I have. Several times. Great movie!

[@RubenKelevra]

Well, this doesn't work anymore, the storage needs exploded in the recent years because high bit depth video recording and 8K started to be a thing. Much better is a ZFS, which just transfers snapshots to a second location. It just needs to send the additional written data and some metadata since the last transfer and you can get older states from your storage server on both locations. If a transfer fails, it can be resumed or rolled back and you can do tripple redundancy, if you like.

[@ninjanippledog725]

You cant work off of tape back up, thats seriously a different need than what this server muti uses are for

[@hamzah465]

"There's a thousand things I can think of and I'd rather be doing that" I feel it.

[@ethand4912]

The seriousness on Linus's face when he says "None of it" is something that I never saw on his face before...

[@skatei]

Take the raid out and blow in it. Should work, I've had plenty of success using that method with my n64 and gameboy cartridges.

[@3ole2]

Rayman Ghafman XD

[@Unc3]

n64 and you have 64 likes. Coinsidence? Absolutely.

[@urielc918]

Rayman Ghafman idiot

[@MrChriss000]

'QUALITEE' poster.

[@spartan876]

beautiful

[@nickpickerwi7787]

This was some of the most gripping video I've seen from a techtuber. Makes me want to get into setting up networks and servers.

[@breebw]

Don't. 20 years ago it was a fantastic career choice. These days it'll send you to an early grave. For a start, your starting pay will be, equivalent for the time, 20% of the old days. The rest of the time will be holding things together with tape and string, because budgets have been turned down for the 3rd year straight. Lastly, a little saying I heard a few years ago: "Accounts departments love IT departments. Before there were IT departments, everyone hated Accounts. Now there is IT, everyone hates the IT department instead of Accounts."

[@diegouy8277]

I would say pure network administration or network engineering isnt a bad career choice.

[@duroxkilo]

setting up things can be fun, maintaining them is a depressing adventure. there are people w/ ptsd after a few years working with situations like the one presented in this video.. no exaggeration.

[@lauwzboy]

what a nice time re-watch this, can't wait for the new episode!

[@johnnythegeek]

Still think you don't need an antistatic wrist strap?

[@MrStevetmq]

+Johnny Phung (johnnythegeek) I don't think his problem had anything to do with static build up or that a antistatic wrist strap would have made any diffence. The motherboard was failing while the system was running and not as it was being worked on. (sorry for the spelling I am dyslex...)

[@OhhLoz]

+Brian Liberacki Pretty sure you guys missed the joke

[@TeemoForLife]

Static no need to worry about as long as u touch something metal before u start building done. U can create static on carpet for 5 minutes touch a PC component and will 99.99% live. So rare static will damage PC components

[@mop10000]

+Johnny Phung (johnnythegeek) kmsl

[@johnnythegeek]

+OhhLoz Yup. lol.

[@CozyCatte]

It's so funny to watch this years after it happend, look at how empty the server room was back then :'D Also seeing some old faces that don't work at LTT anymore makes me kinda sad :/

[@remigiusznowak7277]

Does this asian looking guy still works there? (Idk most of them as i am not regularly watching him

[@thetalesofdaneandco]

@@remigiusznowak7277 Depends which one you are vaguely describing. Dennis does, Andy does, Jono does.

[@Ave-S]

@@thetalesofdaneandco Well, we did see most of Dennis from the " you only liao once". Also, i will like to add that the amount of pain i have recieved from the Live Laugh Liao sign is far too great to thepoint of it becomes funny

[@questwalkerko]

" Huge, Huge, Devastating Loss if we lose this " Title: "All of our data is GONE!" *_foreshadowing_*

[@milky1234123]

i remember this when it went up, at the time i was making back ups on a monthly bases and only one back up. This video is what made me start making 2 back ups a week and having more then 2 back up of those back ups just in case. I know to well how much losing data like this can hurt and make you cry

[@Ken-vl4wk]

At least you guys had the balls to tell the world you used RAID 5.

[@Choice777]

is raid 5 bad?

[@keonfarley]

+Choice777 yup

[@ManzoAlexis]

It's cool to move large files but you are risking loosing al your data, is like a house of cards.

[@eksadiss]

Not true. Raid 5 uses parity so if you have one drive fail, you can swap it out and the data will be rebuilt on it.

[@MD-vs9ff]

eksadiss RAID5 is dangerous because if an unrecoverable read error occurs on any drive during the rebuild, the entire array is lost. When rebuilding, every bit in every drive needs to be read to recalculate the parity. With modern hard drives being so large, UREs become much more likely. Because of rebuilds being dangerous, RAID5 is the worst redundancy method in the bunch. Having a hot spare actually makes it worse since that automatically triggers a rebuild before you can take your most recent backup.

[@TheTechChef420]

who here is watching in 2018 becuase you love seeing linus freak out.

[@jacketylon]

Jeffrey Tripp me!

[@nickpickerwi7787]

Me, lol

[@captainredneck0683]

Me!

[@mkroshana]

😁

[@uliriamu]

Me, it came up on my recommended. So why not

[@nordemoniac]

I'm glad this video is still here. It's important for people to know that RAID = Speed and uptime (redundancy), NOT backup. RAID 5 has a major flaw with our "giant" hard drives today. I had a 4x2TB setup (software) and when one drive failed, it tried to recreate so many times, that another one failed before it could finish = all gone. No problem though, purchased new drives and restored 5TB from offsite backup (friends house). I actually replaced my 4x 2TB in RAID5 with 2x3TB in RAID0. It's cheaper and simpler. Of course if one drive fails, it's all gone, but doesn't matter because I don't need the uptime. But I also have weekly backup on a 2x3TB RAID0 NAS, and constantly synced cloud backup at all times. NEVER confuse RAID with backup.

[@nordemoniac]

Dihelson Mendonca JottaCloud. Approx $80 per year and unlimited storage/devices.

[@nordemoniac]

Dihelson Mendonca Yes, unfortunately it looks like you get max 6-7MB/s. It’s fine for my use, as it syncs immediately and I don’t edit my files too often. The price is unbeatable though.

[@nordemoniac]

Dihelson Mendonca Oh wow, that’s horrible! 6-7MB/s would be approx 50-60Mbps (even though it’s 1/1Gbit here.)

[@CevelNet]

Sorry for coming back to you after 4 months. But I think the take away message here isn't "RAID5 stops working in 2009". One of his RAID cards corrupted due to a bad (consumer-grade) motherboard, so all the drives hanging at that card got kicked out. Only a RAID-1 using atleast 2 RAID cards could've survived such an event, as this is equal to 8 out of 24 harddrives disconnecting at once. Enterprises use multipath SAS to mitigate such events. To not overextend this post, here is my list of takeaway messages: - backup (3-2-1 scheme) - never use hardware RAID cards - use enterprise-quality gear instead of "gaming" motherboards for your business-critical data - don't mess around, if you don't have any clue and get help quickly (like Linus did) That beeing said, I think you shouldn't do RAID at all. As you said, RAID is for speed and uptime. As you mentioned you don't need uptime and modern HDDs saturating gigabit NICs, I don't see a reason to use RAID0 in your case. Instead of losing ALL data in case of a drive failure, you'd only lose data on that drive and recover it from backup.

[@javaman2883]

GlennJacobsen You backup to a RAID0 NAS?!? That seems risky, either of those drives fail you've lost your backup. Well, I guess you also have your off-site backup so maybe it's not too bad. Years ago back in the 10GB-30GB drive size days, I had a drive fail in my desktop, then found out the USB external drive had also failed. I lost so many pictures and documents. So now I'm a bit overboard with backups. I use BRTFS RAID 1. It's not quite the same as traditional RAID 1 where every block is mirrored on both drives. It's a little more versatile, it can use 2+ drives of any size, it always ensures there are 2 copies of every block spread across the drives. I have two 1 TB drives, one 1.5TB, and one 3TB drive. When one drive fails, it does not have to rewrite everything to a new drive, but only needs to write blocks that have only one remaining copy. A second BRTFS RAID 1 array of one 4TB and one 5TB drive is used for my backup. A nice thing with BTRFS is I can bring up the second array, issue a BTRFS send command, and it quickly backup up only the files that changed since the last send. This makes backups faster because it does not have to do date compares and calculate checksums, it just compares the integrated file checksums. All my pictures and videos are also copied to my wife's desktop, as well as backed up to Google Photos and Flickr. My important documents are backup up to Google Drive. I don't backup my OS to offsite, with Linux I can reinstall from a USB live easily. I triple boot, Windows 10 and two different Linux distros anyway.

[@maxtornogood]

"And we'll have a better backup scheme in the future, I promise" Petabyte Server in 2022: "Yeah we need to talk Linus."

[@icecreamjunkie6790]

I work in IT and these types of videos remind me to be grateful that we migrated to the cloud for storage (and basically everything else except one domain controller, which we’ll be getting rid of soon in favor of full Azure AD).

[@derekpohl]

"Wait we can recover some data!" "really?!?! what did you recover?!?!" "A RAID SHADOW LEGENDS SPONSOR? THATS ALL WE GOT BACK???????"

[@ZackHab]

Raid:5 shadow legends

[@jm56585]

Nice transition

[@Nunya58294]

Can you not with that bullshit

[@kasenratliff1832]

this comment aged well

[@vasilicateodor1862]

@@ZackHab LMAOOOOO