These Servers are TOO EXPENSIVE - Hybrid Storage Explored

[@LinusTechTips]

The Apple fusion drive bit is a gag I should've probably presented better as a joke, lol. Basically, a Fusion Drive was Apple's take on the technology and I just thought sliding that in there would be comical. Unfortunately, not everyone knows that. Sorry for the confusion. :P -Editor

[@nova-man]

hi there, forward this to linus please thanks. www.c-sharpcorner.com/UploadFile/cd7c2e/how-to-set-caching-options-for-a-shared-drivefolder-by-usin/

[@TheTeuthras]

Thought it was true at first, but its Apple. A lot of what they do is just marketing jargon of already somewhat established technologies.

[@chiefbigpooh]

I did lol irl when i saw it.

[@devvielife2968]

Haha

[@devvielife2968]

Linus Tech Tips A better sponsor for this video would be honey for saving a buck

[@bluesmachine1006]

Linus: “so I’ve got a problem” LMG Staff:

[@mayrln]

r/programmerhumor

[@default632]

@@mayrln ihavereddit :)

[@wilsonkff]

Let's drop it.

[@results4526]

This should have been the ending sponsor spot: Linus: "to take some of the sting out of things.......TING!" not Squarespace :P

[@klm1784]

Linus makes editors more efficient Editors make videos for viewers Viewers watch Linus make editors more efficient

[@joojoobeans8671]

Please use commas

[@Calihan]

@@joojoobeans8671 Three separate lines, why do you need commas for that? When you drop a line it's already a pause.

[@touchofthorn1841]

when your that pathetic to correct a grammar mistake on a KZbin comment that isn’t actually a grammar mistake

[@xicofir3737]

Linus should water cool the editor's for better performance.

[@franscartoons]

8k because future generations need to watch Linus drop things

[@franscartoons]

Manuel Kreuzberg pewdiepie uses a broken super8 camera

[@bryncb]

Main reason they record 8K is probably to be able to crop and maintain 4K resolution.

[@Steamrick]

Plenty of reasons to go 8K regardless. Cropping without quality loss has already been mentioned, for one. Also, they want good quality footage available for the rare occasion that they put in a flashback moment years from now. Besides, plenty watch at 4K. I do, for one, despite my 1080p screen, because with youtube's extreme compression it gives better overall quality when downsampling. And I might not go back to watch a review of some kind in 10 years, but I very well might go back to a few miniseries like 'honest answers' or 'the rugged, manly way' that'll still be interesting.

[@lmmgamesandmore3296]

@@Chris-rg6nm wtf i watch everything i can in 4k

[@idcrafter-cgi]

Jea too see all the Details

[@techgamexstudio.6858]

Pulls apart 40k red camera Droped 10k processor Have to buy 7k worth of used SSD This year is going to be hard for Linus....

[@VanlockFR]

ouch, when did he drop a 10k processor ? :D

[@saifudinkhuzema6525]

@@VanlockFR 100k build

[@OtherTheDave]

Did the camera not work after it was reassembled? I thought that it was still taken apart at the end of that video.

[@manaspradhan8041]

@@OtherTheDave I don't think they have reassembled it yet

[@CookedPorkchop10]

@@OtherTheDave They haven't reassembled it yet, as far as I know

[@dazr6604]

You need to run the Tier Optimization Windows Schedule task to have Storage Spaces use its heat map and move the data between the SSD and mechanical tiers. You totally forgot, or didn't know, to do this. You can also specify the file name extensions that will always go to the SSD tier by default and how many days they stay there for. This is all built in, but you didn't set it up!!

[@CaffeinatedTech]

Are these things obvious in the GUI, wizard, or quick-start guide; or are Microsoft still trying to sell certification training? I mean these steps sound fairly important to a successful setup.

[@andrewyork3869]

@@CaffeinatedTech just another reason to use Linux....

[@CaffeinatedTech]

@@andrewyork3869 Exactly.

[@andrewyork3869]

@@CaffeinatedTech we need more open hardware

[@ZerdN]

Isn't the default to fill up the SSDs first though?

[@xilefhd]

Why don't you add more RGB lights to the Server to speed it up?

[@youtube_gaming]

Yes light is speed. Speed is key

[@leonflpqzhz4765]

You're wrong RGB lights only speed up processing speed not reading and writing speed.

[@CheesyPencil]

Leon Flp Qz Hz they do both don’t you know the lights hit the hard drive / SSD or anything and speed them up and if the light are on them it’s even faster!

[@Spr33h0x]

That's not how drives work... You need to slap a spoiler on that bad boy

[@wbblueye1387]

@Just Pizza Open Door = Open Door

[@TheCatpirate]

You know it's bad when Linus calls something expensive.

[@zenv9180]

Uh no he always calls things expensive lmao

[@themonroe654hd6]

It's lip service because all this stuff comes free anyway.

[@Bierkameel]

No he is just cheap and using toys to keep a company running.

[@tggasser]

Expensive? This dude has no idea. You should see how enterprise grade storage servers deal with such a small workload. Perhaps he is a KZbin "Tech" reference, but in the industry this would be called child's play

[@oldschool1079]

Lol, so this how it looks like when Linus goes *budget* :)

[@InIMoeK]

Storage spaces in 2k12 works with a scheduled job for tiering. Try S2D in 2k16. That wil scale

[@thomawesome90]

This is a nearly perfect answer server 2019 interfaces with the new admin center much better to give better reporting on cache etc.

[@JonasSkullerud]

S2D in 2016 is pretty awesome. Hitting a million IOPS on random tests. Its also not per file, but per block so only parts of the files is cached if needed so (ie parts of a vhdx file (or movies in Linus' case)).

[@thomawesome90]

Last year in September 13.7 million iops demonstrated with s2d. If LMG expect enterprise grade performance to match the professional production and delivery of content money has to be spent on the correct grade of hardware for the job. It's great to make a video using consumer grade storage and a consumer based storage software, but surely now it's only fair to do storage spaces properly with a real s2d solution?

[@InIMoeK]

thomawesome90 I also noticed that a 2 node cluster is now supported. And dedup on ReFS. Only down side is that you need datacenter licensing which can be pretty expensive

[@Thefreakyfreek]

Why write 2k12 when 2012 HAS THE SAME NUBER OF KEYSTROKES

[@xorinzor]

- Doesn't wanna spend 7-8k on more SSDs - Takes apart a >$40k RED camera to make it water cooled LTT logic.

[@marcusm5127]

Buying ssds isn't cool opening a red camera is. It's like deliding a xeon platinum VS just buying more.

[@tyriekcarr9123]

Builds a $100,000 pc

[@MaksKCS]

@@marcusm5127 Oh yeah, because doing "cool thing" is more important than not being worried about destroying an 8k camera. _fuck im probably gonna get wooshed_

[@arbitraryalias9825]

-Has 20 employees and is hiring more. -Doesn't want to invest 8k to make them more productive. Startup logic.

[@kernelpanic2887]

@@MaksKCS _As you haven't got woooshed yet._ r/woooosh

[@djdannypachy]

Hey Linus, what you mention is possible to do! You can use your existing servers (or file shares). Basically, you would have two seperate file volumes / file shares, one would be using NVMe SSD drives and the other would use hard disk drives (your archive). You can create a PowerShell script (or find one online) that would move the files (i.e. project folder) from the fast to slow storage based on your criteria. For example, if a project folder hasn't been accessed, modified or moved within a certain period of time it would be moved to the slower storage. This can also work the other way too, if a folder has been accessed or modified in the slow storage (your archive) it would be moved to the fast storage. You can have the Powershell script(s) run automatically using Task Scheduler. For instance, you can have one script that checks your fast storage on a daily basis, and another check your slow storage more frequently (every hour, 30 minutes, 5 minutes, etc).

[@vovochen]

So sad he wont read this :(

[@VideoInformation]

Interesting

[@Llama052]

Or just use a real caching method that isn't Windows Server lol

[@Kieeps]

But that would get you 2 shares, the good thing with cached systems or layered tiers is that it will still only be one share even though its on the fast, or the slow drives... There are employees and LTT that doesn't know what a share is :-P that itself would yeld cunfusion/social bottlenecks that's way slower then the time you save on the system itself :-D social engineering at its best.

[@houdwarz]

@djdannypachy's Scheduled Task idea (or file copy triggered task even ?) is an interesting solution, but this can be simplified to a single File Share, using Tiered Storage and Tier Pinning (ps> Set-FileStorageTier) based on files last access times, combined with forcing an immediate Tier Optimisation to apply the pinning changes aka move files from tier to tier (ps> Optimize-Volume -DriveLetter X -TierOptimize). Your Virtual Drives would also probably greatly benefit to be setup with a large Write Cache, which can only be customised using PowerShell commands. Lastly, to be honest, learn PowerShell if you want to play with Storage Spaces ! You can build yourself a script to easily destroy and rebuild the whole Storage layers and File Shares in a minute using different parameters that you can then benchmark for your specific use cases. Several parameters are also only available to the Powershell commands.

[@ruediix]

As a note, Linux has several semi-automated solutions other than the old standby ZFS. There are several block layer options such as BTRFS tiered storage mode, BCacheFS and LVM Cache mode. BCacheFS is considered still beta, and I'm not sure if you'd be comfortable using it, even with the stable feature set, on data. There is an older project that only implements the block layer transposing it into a unified virtual block device, but that would have a steeper learning curve and more complex setup. LVM Cache mode has a rather high learning curve, but there are GUIs to set it up that make it easier. I set an LVM Raid up in the early 2000s using such a GUI it wasn't that hard. Still, it's a steeper learning curve and not an automatic solution. BTRFS is an automatic solution which is nice. It can even handle automatically handling the desired redundancy data across drives, and whether to bother with redundant read or just redundant write. You would want 2x to 3x redundancy on the permanent storage with redundant write and interleaved read. For the upper tear you probably want full interleave across all devices. BTRFS was designed by Oracle for large objectbase files, because of the exact ZFS issues you encountered. It works well with all sized access from BTRFS. There are also several options utilizing the VFS layer and a FUSE transparent conversion. In this case you would want EXT4 or BTRFS on your physical layer managed by BTRFS and F2FS on your SSD which should run a conventional interleaved RAID. You want write-through cache or a quick flush cache. The later would give better performance, but the former means near zero chances of data loss from power failure to server. In all these cases Linux gives some pretty good options as of cache management. You would probably want simple Most Recently Accessed for your purpose which is much like an FIFO except it gets pushed to the back of the queue every time it is accessed. You will likely either want a short flush cycle or write-through. A short flush cycle will greatly reduce write times with minimal potential for data loss in a system failure of the SSD layer which is tuned for speed not reliability, while a write through will insure insure immediate copy to the long term layer, but at the cost of substantial write speed and increased fragmentation of the physical layer. (BTRFS introduces means to reduce fragmentation that weren't in ZFS, but they work even better with a short flush to disk wait.) As of getting better performance, on any OS, don't use mirror on the top layer, and use a high redundancy single read, mirror on write system for the long term storage layer.

[@11wallace11]

Good luck getting Linus switching to Linux

[@slepix]

Storage spaces uses a regular scheduled task to "optimize" the volume. You can find it in "Storage spaces" section of task scheduler. If you execute it, it will analyse the usage and promote hot block to ssd tier. There is an option to forcefully promote a file to tiered storage using powershell. It's one command iirc. When you create a volume via ps, you can specify the size of WBC which will improve your writes. Default is 1 GB which means that the first 1 GB of data will go to ssds and then spill over to spindles.

[@slepix]

Also, there are couple of options you need to tweak in order to get SSD/HDD cache working properly. By default Storage spaces doesnt use SSD/HDD built in cache.

[@Nathan-gj8ch]

This is how I setup my 30TB plex server so making thumbnails for new incoming videos is not slow and the whole extended family can watch the new movies simultaneously with out queuing or buffering

[@raffaelezippo8109]

@@Nathan-gj8ch Such a complex high performance tool used for home streaming from a Plex server. Relatable

[@TheIvicask]

@@slepix what do you mean it doesn't? If u create storage pool containing slow and fast ssd tier it automatically uses faster tier to store hot data, with resfs it does even live tiering, and write back cache allways works but it's set to 1gb default unless you set it bigger via powershell

[@TobiasTimpe]

Forget it, he already gave up and filled the SSD server's second row ;)

[@tarekl.3725]

the hidden purpose of the video is probably to get someone to sponsor this project... 10/10 would have tried the same if i were linus' sandals and socks

[@NinjaInTheFirstDegree]

Surely he has enough hardware lying around to build a proof of concept.. then spend money.

[@tarekl.3725]

@@NinjaInTheFirstDegree it's not just the hardware....

[@TerranceShepherd]

Float plane video already revealed they just bought more nvme drives.

[@NinjaInTheFirstDegree]

I understand but its just windows server, which can be implemented for free temporarily. So any money is going into hardware at this point.

[@NinjaInTheFirstDegree]

And the main question is how was he only getting 220MB/s from 4 10TB and 2 SSDs? Raid 10 with those 4 should have gave him at least 400MB/s I sure hes was using 10gbit.

[@mikethewordsmith4263]

Watching Linus walk around with a mechanical drive is leagues better than any horror-game streamer.

[@i-win]

This means I can't run my Minecraft server smoothly?

[@fazeobama8872]

@Fun for CHEAP HOW CAN PEOPLE ALWAYS NOT GET JOKES

[@Temppy1]

@Fun for CHEAP i think his memeing

[@CaveyMoth]

This means I can't clean my metal computers properly?

[@patience__8051]

@Fun for CHEAP now try to run 200mods on said server

[@OGBeefStew]

Are you cheating on my server?

[@jasb78]

Linus... Storage Spaces Direct basically builds a SCSI adapter not too different than any other storage adapter in device manager. This means you can stack various other caching technologies on top of Storage Spaces to build the type of tiering you need. You need a minimum of 2 ssd/nvme cache drives in mirror mode per pool so that if one cache drive dies the other one can still take over. The Cache drive must support write protection during a power failure event (a battery backup on board the cache drive) Also, consumer grade nvme and ssd drives are not suitable in storage spaces direct because they don't support power failure protection. So once you have a Storage Spaces adapter defined and built you can attach tools to it such as "Fancy Cache" or "Condusiv Diskeeper" to prioritize hot data in RAM, eliminate fragmented i/o., eliminate disk fragments, save wear and tear on your disks, etc. Your biggest worry will be ensuring the battery backup can hold up long enough to flush the cache; OS crashes are also going to cause data loss as well. Also you're using an outdated version of Storage Spaces (2012) and the 2019 version of Storage Spaces Direct is a much improved overhaul of Storage Spaces. Caveats of Storage spaces is that you cannot shrink or expand fixed sized pools, and if you run thin pools you had better ensure your enclosure can physically house enough storage in the future to cover the thin capacity you specified. Your best use case with storage spaces is to run in a cluster of at least 4 nodes, if you intend to run in thin mode. Storage Spaces supports expanding across multiple nodes so clustering is its best use case. In a clustered Windows Server environment your licensing costs will still be significant. Also, consider mirroring across entire nodes instead of just within one node.

[@NicolasBana]

Real MVP. I understood maybe 75% of that, but damn storage technologies are cool

[@smallfoxx]

Also, rather than have a huge cage of drives and a single server, you might consider doing a Storage Spaces Direct across multiple servers with large drives and cache in each. That way you can minimize your bottle necks and scale out additional nodes to increase performance rather than just trying to add disks. docs.microsoft.com/en-us/windows-server/storage/storage-spaces/storage-spaces-direct-overview

[@nybrand]

Thanks for the url

[@TK199999]

@@smallfoxx There is a possibility if Linus uses multiple severs, that networking bottlenecks could occur. Also I agree with others Linus may need to bite the bullet and switch over to enterprise level system drives for his video's. I am not an expert, but I think one of the big issues Linus is running into is file size problems. Meaning his 8K video files are just to large for his consumer level setup. I suspect in the future these problems will only get worse and file sizes will just get larger over time, so Linus may not have choice in the end. I bet with a little work Linus could get a sponsorship from one the enterprise drive and server system companies. To pay for and install a new enterprise system, just have Linus participate and watch as he basically drops things, along with other Linus shenanigans.

[@smallfoxx]

@@TK199999 He's been putting a lot of money into getting his 10Gbps everywhere network, so spreading the load between multiple hosts with 10Gbps each should help to utilize the available bandwidth between it and the guests. As for large file sizes, it was designed to be able to work with VHD's for VM's and SQL databases, so it should be able to cope with it, but mileage may very. However, when it comes to hardware, I agree biting the bullet and moving to enterprise is likely where he's at; there are lot of options out there.

[@BlackBird26]

your employees are pretty lucky if something is slow you make a video on making it better for them, the rest of us just have to deal with shit technology in our businesses cause its TS lol

[@meloD30]

Wait, they give you technology at your job? **drops chisel and mallet**

[@zakirmughal4027]

This guy is cool he should start a KZbin channel

[@barthilvandeelen]

Zakir Mughal Oh you are so dumb! He is on youtube! R/woooosh

[@frederikmolenkamp1850]

Perfecte Gast iTs A JoKE LOl uR StuPID

[@niklas8135]

Good idea!

[@LA-gx9mh]

@@barthilvandeelen is this a whooshception

[@MrMrRubic]

@@LA-gx9mh i think he is ironic. I *hope* he is ironic.

[@klink388288]

You can accomplish all of what you were describing on linux. Namely: unionfs to make your ssd pool and hdd pool showup as one and the same. Cronjob with "find -atime ..." that runs at a desired interval and moves things around based on last access time

[@St0ner1995]

Linux: the answer to the worlds problems

[@VidKa0S]

Like the above comment so that Linus can read it

[@aanesijr]

Yeah but that locks I/O during the move. You need a lower abstraction layer.

[@klink388288]

@@aanesijr a non issue really with their use case. Files wouldnt be moved off the cache if they haven't been accessed for x number of days so in the off chance they tried to write exactly at the time of the move, then their op would wait the few extra seconds. Union-fs supports copy-on-write in the other direction. Plus linus said keep it simple lol. Literally easier than the powershell commands linus was running.

[@klink388288]

@@DihelsonMendonca thats why i listed the tools and their usage

[@zcixoT]

Tiered Storage Solutions are not meant for Video editing... You could technically use it in the correct setup if you have security cameras with a constant data-flow that cuts the files into say 5GB segments. If you then need one of the files it will bump it up to Tier 1 and go from there. But working off of a system like that using the SSD Tier as a cache instead of an actual storage space which it should be seems like an extra step that just adds workload to the storage/server and reduces performance. In the Consumer-market there used to be SSHD drives which were HDDs with an SSD cache built in. Obviously didn't sell up too good though lul Most data tierings won't work too well with large video files since well... they are large video files without compression. then it's 8k footage which just makes the whole thing explode and puts a gigantic workload on the server/storage that has to process it.. The Tiering technique Linus is trying to implement could work for the basic little company that has all their Database-stuff as simple documents and that can be compressed without a problem. I don't even know why I felt the need to write this excessive comment but... well here you go ^^

[@MikeDawson1]

this is the answer. 300GB compressed files that are hit with heavy random IO won't work well with tiering. He should test building out server with a huge array of cheaper SATA SSD drives instead of NVME if he wants to save money - if you throw enough SATA SSDs in there they might not notice any difference from the nvme for video editing (would be good to test out in his case)

[@zcixoT]

@@MikeDawson1 that is true. But in the long run.. we're taking about Linus he's gonna buy the expensive drives anyways lul

[@joshs3229]

Excessive comment or just an uncompressed comment?

[@HardwareNumb3rs]

Actually this is the right comment for a video like this, for video editing workloads a storage with 8+ sata drives and some nvme as cache works way better than tiered, and less expensive too, QNAP 8/12 bays are great for that, and cheap (against enterprise storage competitors)

[@HardwareNumb3rs]

@Nick McCollum NVMe is a terrible waste of money for video editing, there's no need for high IOPs like a Database or VMs, even an array full of SATA SSD is wasted, for editing they need capacity and throughput, 8 disks in raid 5 can easily saturate a 10Gbit LAN, that is the ultimate bottleneck. A 12 bays QNAP rackmount Xeon based full of 10TB Seagate Ironwolf drives will cost arount 7K$ for 120TB of RAW capacity, 58$ per TB, try to do that with NVMe or SSD...

[@atroxes]

I suggest you take a deep-dive into ZFS on Linux. Without knowing the specifics of your requirements, I'm fairly certain it can achieve pretty much what you want with a decent amount of RAM and if you tune your L2ARC depending to your workload. You could potentially also script your way out of reading newest modified files regularly to prime your L2ARC if necessary. Regardless, you're going to end up with an "almost there" solution, unless you spend some canadian fun-money and go full NVMe :-)

[@josephblasi7674]

full NVMe with amd cpus so you are not being capped by intels limited pci-e lanes.A+ Server 2113S-WN24RT

[@JustTechGuyThings]

@LinusTechTips Try ZFS's L2ARC. The default is just to keep things that are used often in ARC (RAM) while L2ARC (SSDs or NVME) catches anything that falls off with the idea that things that are used often get dropped into L2ARC. I'd recommend Mirrored Drives (SATA SSDs), then NVME for L2ARC and then as much RAM as you can. Don't forget to set the fill rate on the L2ARC as high as possible and assign as much RAM as you can spare.

[@jr_kulik]

Small Canadian man realises he can’t save money.

[@turing6976]

upload this video to pornhub with that title

[@karnige5804]

small canadian man has attractive asian wife, own multi million $ business and lives in the best country in the world

[@jr_kulik]

The Karnige “Best country in the world” ha you haven’t seen Switzerland then. Like Canada, but much better.

[@remytv]

Would be a great slogan for picking a cell phone plan too. Lol

[@Corei14]

Small Canadian man gets FUCKED in ass by realizing he needs more storage

[@MeAtHome5]

You need to get Wendell in again to teach you linux.

[@tolpacourt]

I'm sure there is a FOSS solution to this problem.

[@danieledwards3376]

@@tolpacourt bcache :)

[@ФелканЛавер]

Linus*

[@rudysal1429]

@R3tr0hax Media Group Anthony is cool but wendell is more experienced

[@deViant14]

@@danieledwards3376 apparently also lvmcache

[@codec862]

Buys $50k camera and casually tears it apart Refuses spend 6k on drives to run their business

[@DraconianDebate]

But that wouldn't give them a new video!

[@1stockpunto866]

*DEBT*

[@derSkedda]

12:42 ...

[@derSkedda]

@@HelloKittyFanMan. Its a timestamp where linus says he'll just buy the fast storages

[@romainsavioz5466]

@@HelloKittyFanMan. what point?

[@MrAbdullah5010]

Just add some butter to hard drives. there will spin faster

[@thisisnotmatter]

Btrfs you mean

[@Amipotsophspond]

"even if you rub the engine with cheetah blood." is the reference you are looking for.

[@Trident_Euclid]

Wouldn't they slip and break tho?

[@weirdo6821]

Is that a life hack in one of those bad life hack compilations

[@electronJarvs]

Turned my hdd into a ssd, thanks for the tip man!

[@curious_crypto_chris]

You're cheapest option is most likely Debian running ZFS (1GB RAM for every 1TB of Storage sounds like a lot, but there's tons of used server grade ECC RAM out there for sale). Using linux will allow you to get more granular control over all the OS components. Make sure you tweak the ZFS settings for your setup, hdparm to max write speeds, and configure NFS for parallel writes. Another, but slightly more expensive solution would be setting up a distributed file system cluster like GlusterFS which load balances read/writes amongst servers in the cluster to maximize I/O. Depending on your team's file read/write habits, this may end up giving you better performance. The much more expensive but performance oriented solution would be to purchase a used SAN solution. Similar to your DAS, you could purchase one or more EMC/3PAR shelves containing various 7.2k-15k disks and another shelf containing only SSD drives to be used as flash cache. If you want to go all out, full enterprise: 1x Storage Rack consisting of several large storage disk shelves, 1 or more fast cache shelves, 1/2x SAN switches, and Fiber channel cards for each desktop. It's expensive, but you can get insane performance this way. Broadcome and various cisco manufacturers support SAN speeds of up to 128GFC (204Gb/s). I love watching these server tour videos. Keep 'em up!

[@halflife82]

CryptoChris I 100% agree. Even before watching this video I was asking myself, why is he not going with enterprise solutions??? I work for a very large company in IT and the needs of his File System is tiny compared to what I’m used to. There are so many options to increase his thoroughput, though some are expensive I will admit. I just kind of laugh that he’s trying to do all this san workload with a “desktop grade” solutions. Lol

[@costafilh0]

So he needs 1024Gb of Ram? cool!

[@kaldo_kaldo]

"1GB RAM for every 1TB of Storage sounds like a lot" Honestly not really. They have 24TB of usable space on their current setup. 24GB+ of RAM is frequently found in a lot of above average workstations, and higher end desktops and laptops, let alone servers. Of course, the goal would be to vastly increase the storage space, but if they went up to... say 256TB, then 256GB what they would need, and it equates to a decent server, it's not even a high end server. You're looking at about $1500-2000 for that, which compared to the upgrade he might do here ($5000+) for maybe getting closer to 50TB, sounds like quite a bargain. Though of course, the 50TB option has no drawbacks beyond price.

[@AfifAhmad]

What you've described is exactly how his archive server(s) are set up using ZFS and GlusterFS. Check out his Petabyte Project video

[@fuckelonmusk]

@Lassi Kinnunen "Linus Media Group is a full service production agency with decades of collective experience in web video and social media and influencer marketing. We've worked with brands as small as cottage-industry one-person operations all the way to large multinationals like Intel, IBM, and Cisco."

[@TimvanHelsdingen]

Don't recommend windows storage spaces. I ran one for a year (with caching ssd's) but what happens is that after a while you start to 'lose' space. It will misreport the data used. I had about 12tb in the pool, i was using about 5, but after a year it started complaining it was almost full, the data reported as being about 2-3 terabytes bigger then it was. There is no way to fix this, and it's been a known issue for years. I had to move over all my data, remove the spaces and i changed to regular raid setup after that (which is what I'm using now)

[@TimvanHelsdingen]

@@johnstrad I used windows storage spaces (what linus is using in the video, and why i am recommending against it) The issur i ran into is a issue microsoft has known about for years and they haven't fixed it. There is indeed a rebalancing tool but it doesn't regain the lost space. For as far as I could find there wasn't really a solution to it except for removing the storage space and starting over.

[@TimvanHelsdingen]

@romaneeconti02 You're aware you can also speak normally to people on the internet without insulting them right? Anyways, there isn't really a whole lot of configuration you can do to storage spaces by default so it's not really something you can misconfigure to the point that it would misreport data used by 30%. Also, you really think I didn't spend a long time googling this issue? I tried a ton to get this fixed, including a whole bunch of powershell stuff. But there's even a whole bunch of posts about this on the Microsoft forums and not a single Microsoft person that knows how to solve it.

[@NathanielHatley]

I've been using Windows Storage Spaces since Windows Server 2012 R2 came out and haven't had any significant issues besides my ReFS volume not mounting on bootup sometimes which a restart fixes. However I only use fixed size vDisks.

[@TimvanHelsdingen]

@@FujiLivz Isn't that what the 'optimize drive usage' buttin in the windows spaces GUI is supposed to do? That didn't do anything though ( i ran that about weekly) But i'll investigate some more. Windows storage spaces was pretty nice overal but these issues were really of putting.

[@jwaffled4143]

"Too expensive" **Builds 100k gaming pc**

[@SolarShado]

Pretty sure most of those kinds of builds are sponsored. That is, they get sent most/all of the parts (or at least the most expensive bits) for the purpose of showing them off.

[@mayrln]

@@potato_x69 woooosh to you good sir

[@mayrln]

also fuck your long ass name

[@RS-xx8gz]

kaan turk

[@morpheas768]

He didnt spend 100k on a gaming pc ffs. Are you insane?

[@alvinchan7746]

ssd config cost 8 grand linus drops a priceless xeon... 🤔

[@suborgtfo.4433]

_Wallet server has left the chat_

[@WarmEyes25]

wallet.exe has stopped working

[@frassefrazer]

Linus: ”Mirror mirror, on the server wall - who is the fairest one of all?” Mirror: ”Stephen Burke” aka Gamers Nexus

[@ScammerRevolts]

Rip the wallets money.

[@AholicKnight]

Hi!

[@dublowduck7823]

I would LOVE to see a follow up to this where you try different programs or approaches, really great video!

[@joflo5950]

I love your server videos

[@k1ngjulien_]

I've been watching ever since the whole room water cooling days and I have to say that their history with servers is really what made me interested in this topic. Today I am running my own pfsense router, private server and remote server and I am glad to have been inspired by linus.

[@joflo5950]

@@k1ngjulien_ the watercooling thing was damn cool

[@k1ngjulien_]

And you might be interested in this :) : Mark Furneaux kzbin.info/www/bejne/mmK2mJxpnbZ0as0 cnlohr: kzbin.info/door/G7yIWtVwcENg_ZS-nahg5g hak5: kzbin.info/door/3s0BtrBJpwNDaflRSoiieQ lawrence systems: kzbin.info/door/HkYOD-3fZbuGhwsADBd9ZQ and many more!

[@k1ngjulien_]

@@joflo5950 I agree. They should try a v2 with the knowledge and tools they have today.

[@jafizzle95]

Same. I bought unRAID and got into servers because of LTT's server videos. These server videos are my favorite ones.

[@kaldo_kaldo]

I always appreciate your server and server-grade related content. There aren't a lot of people showing this sort of content to the public, but more importantly, no one is really doing it in an accessible way like LTT. I'd bet that 80% of your userbase can build a computer and they're not really here to learn about how to do that but rather to see interesting tech that they'd never get to use themselves. The server side stuff is among that, but it's also an opportunity to learn and see this interesting side of computing that's usually only talked about in small, confined spaces with loud fans.

[@hugemad]

I'm going to be that guy but whatever The default Administrator account on Windows Server should be disabled and a second admin account should be created and used instead. The default Administrator account doesn't have UAC prompts which is unsafe for the security of your systems and entire domain.

[@andrewyork3869]

In security if there is a wrong way and a right way to do something, count on it being the wrong way....

[@leftyeh6495]

The right way is so damned annoying everyone does it wrong.

[@crzr5]

This was a test server, is not their main server

[@StolenPw]

@romaneeconti02 Are you telling me you're using UAC on your PC right now? Every time I see somebody with a UAC popup I automatically just assume they are not literate enough to understand how to even turn it off which in that case its better turned on as its idiots like that who need it most. You're also implying their server would be a target for anybody. Vancouver's hacking scene is a joke let alone anybody with the ability to break into his system finding it worth their time. Nobody is going to risk going to prison just to delete some youtubers videos off a personal sever.

[@Kenneth_James]

@romaneeconti02 @stolen password has left the chat

[@seafoodsalad3356]

Always funny but never take these guys advice for Enterprise applications 🙈

[@jackielinde7568]

Well, most companies can't afford to have their IT teams experiment with hundreds of thousands of dollars and staff productivity time. (Yes, I have lived through such an experiment in the late 90's and early 2000's while doing IT Service Desk work. It's not fun for anyone.) So, when a small tech company, like Linus Tech Tips, is willing to do said experimentation, it's a good thing. My suggestion is take their advice with a modicum of salt, and do your homework. (AKA never rely on one source of information for an expensive answer.)

[@frostyphoenix1278]

@@jackielinde7568 Agreed. That adage applies to everything though: One source is almost never good enough.

[@backupplan6058]

Especially when it’s a company that relies on sponsors that can colour advice. Note how StoreMI never even got a quick mention.

[@armybear2]

That is why most companies hire IT contractors to manage their infrastructure instead of doing it internally. Contractors might cost more than hiring a dedicated IT personnel in your own company but the difference is that you don't have to pay for all the research and development cost yourself when trying to deploy a scalable solution. Plus the contractor will usually be made up of more specialists rather than some poor person who has to learn a thousand things that they will never become a master of.

[@emmy4691]

Yeah, it's some cool server hardware and all..... *But is it RGB Fusion 2 compliant?* /s

[@a1ph4n3rd]

>Doesn't want to use a Linux solution because it requires in intimate knowledge of Linux >Tries to use WIndows Server and fails because he needs to run a Powershell command which required help from someone with intimate knowledge of Windows Server just do the needful and get a dude that knows Linux, you have enough servers to warrant it

[@tejaspadhye]

Hire someone than paying 8k. Basic Linus, Basic

[@DaniilGentili]

He may be a good techie geek, but his server videos always show his ignorance in basic aspects like command line usage; whenever you need to work on complex computer systems, knowledge of the platform-specific command line is simply a must; and the worst part is that learning those skills is just EASY, thanks to a thing called "documentation", but Linus can't be bothered to do that, instead he keeps on talking about shiny corporate solutions and asking for user-friendly GUI solutions even in a SERVER ENVIRONMENT. It's just sad looking at him trying to figure out what's wrong with the command the **GUI-aided** command line (that just sounds awful) windows server helpfully provides him with, when most likely the red text the console spits out contains the **exact description** of the issue, and he just has to read some manuals and some documentation here and there to fix everything.

[@tejaspadhye]

@@DaniilGentili EXACTLY! and now do you see how his employees don't give a f about him. Many of his good ones left already..

[@XICwoodXI]

You guys are quite harsh, the video is simply entertainment, Linus has cash up the wazoo! He knows he could simply hire someone who knows Linux very well, but would that make for a good video and would that make his editors any better at doing their job? probably not. You guys act like Linus is running a Multi-billion Dollar IBM type company with Super Computers. He does not need to be savvy with Linux when he already has a pretty damn successful company as is. He simply wanted a little more speed for his editors! He is not building a super computer to map weather projections for crying out loud.

[@TheEpicLinkFreeman]

who left? the guy that ran channel super fun left, and the girl that modeled their videos left.. don't think they were highly qualified to say that he doesn't know anything about servers. Luke was forced to leave so he could work on their other project and be called CEO of that project or something, he still works with/for Linus and in the same building.

[@CubicleCalvary]

Takes apart a Red camera to...see what happens, won't buy a few more SSD's.

[@bigdiglett3258]

i mean yeah that's about as Linus as it gets

[@TheRogueBro]

This is how KZbin works... He is generating profit from goofing around testing ideas. If he's lucky, this one video will generate enough money to pay for those NVME SSDs.

[@dushk0]

@@HelloKittyFanMan. Good to see a fellow man of culture!

[@TheRogueBro]

I'm not sure what you mean by "Which possessions of an SSD were you trying to refer to?". That sentence doesn't make sense to me.

[@TheRogueBro]

Ooooohhh, you're just being a grammar nazi. Cool. I fixed it :P

[@Cyberfoxxy]

"How did nobody at notice this.. for years." Every professional software... ever

[@suhdude9775]

I really like these server videos, can't wait too see more of them. What happened to the 48 port 10 gbit switch you showed us a while ago btw?

[@pierrelezan]

They are going to watercool it ...

[@vincefleming]

@@pierrelezan what could go wrong

[@archerking8386]

Love that apple reference and then an ifix it sponsor.

[@ThatJay283]

7:35 That is why I use Linux.

[@wacknesium]

nobody: linus: *I F I X I T*

[@trush0t1]

wait.. so if nobody said nothing.. that means everyone said something? ill never understand this meme

[@CobaltBrokk]

You guys should do a tour at KZbin and show display how they store their data

[@ThexSinGaming]

KZbin isn't storing 8k video files. They store a lot of data but the size of the files aren't anywhere remotely close to this. They have encoders and compressors for streaming the videos. They have servers all over the world for this and cloud storage. Sure they can store a lot of data but all of it isn't being accessed constantly like the editors are doing.

[@David-bz7pi]

Another route you may want to consider is one mount from another mount. Here's what I'm thinking. As you said, there'll be a time when the video is no longer really necessary to reference, and then gets turned into archived material. Having an automatic job that runs on your fast pool, look for files/directories that haven't been modified in X time (month, etc) and then move the files to a slow storage array and create a simlink/or second share, that the editors can access on the off chance they need it. That way only the important, currently used stuff, is on the fast array. This can create a bit of a clunky interface, so what could be useful is a standard naming convention for your projects, and a file/app/etc that has a listing of the active projects. Your editors can add an old project to the list, an auto job picks it up, and moves the contents back to the fast array. The job, when moving things to the old array, could also update said file/site/etc to show it's been moved to archive. Still requires a policy change, a naming convention, and everyone using that process, but may work

[@adelpozoman]

you should really try linux

[@mohdfaizal6773]

Freed bsd🥰👹

[@Andy-uk9bw]

This guy invented Linux why do you think it's named after him?? Linux Sebastian... hello?!!???! Some people man.....

[@darkceptor44]

@@Andy-uk9bw i hope youre being sarcastic

[@nickc8667]

@@darkceptor44 There is no sarcasm on the internet.

[@username65585]

Yes but give more specific advice.

[@finalbox4416]

LinusTechTips: When times are tough, you can get by with 4K monitors instead of 8K monitors.

[@arnavfernandes8165]

You could just use wohneck as a cache for new projects and then use scripts to sort the new and old footage ,and when editors access old footage ,well their are 2 ideas I have .1)move the footage in optane or to wohneck 2) let them edit old footage off the HDDs as they will probs only need clips and can wait 5 or 10s for it.

[@0xC]

Apple fusion drive. Sponsored by iFixit

[@Maderhase]

please don't drop a server for gods sake

[@usafsteve1079]

Never give up on saving money! There’s a way to do this on the cheap. I know it’s out there somewhere. Exhaust your resources! And most importantly, do more server/mass storage stuff! I love trying to think these problems through with you as you move through the video.

[@grassyloki]

Oh god Windows Storage Spaces...

[@dfarrall]

Linus, all the tech you've complained isn't available actually is already out there.. You just need a proper consultant/employee that knows about it to put it together.

[@jackielinde7568]

I guess Anthony wasn't available for this video?

[@yearswriter]

That is exactly what he said in the beginning of the video: there are solutions, but they are either expensive themselves or require expensive work. Either way he spends money.

[@dfarrall]

yearswriter so instead he’s going to magic together a solution that nobody has ever thought of that requires to skills and is cheap? There’s solutions out there at the right price point, just pay a consultant to point you in the right direction.

[@Etugcu]

@@dfarrall No, the idea is that he tried some cheap "what if" solution and it failed but he did this since it'd be a good video not because he thought he'd succeed and people would talk about it and watch it. Like i thought that is obvious to everyone.

[@SoyDelSouth]

Etugcu yup I agree with this. Obviously this video will help him fund his SSDs. Lots of people getting wooshed

[@nosbig98]

Linus, you really need to talk to the folks at Pure Storage. While their products are typically designed for medium-sized enterprises, maybe they can be convinced to make a product scaled for KZbin creators/video editors. Their FlashBlade product is the closest to what you need. It is a 7U chassis with 15 slots. Each slot can hold flash-based storage in 17 or 56TB of storage. You can start out with a smaller capacity and scale up as you need. Since it's all flash, it's a single tier. Pair that with your choice of network SFP (bonded, even), and you have a turnkey solution. I work in a hospital of approximately 2500 staff and 300 beds. We have about 300 server VMs and about 200 VDI VMs running on one of their FlashArray devices, and we can't get this thing to break 1ms of latency on a 16Gbps FC SAN. These guys know what they are doing and are the most affordable on the market.

[@xilefhd]

Mabye you can get a sponsorship with Samsung for all the SSDs Linus

[@Rainbow__cookie]

This video is bought to buy Samsung

[@jackielinde7568]

I'm surprised he didn't reach out to Intel, since they do both the Optane and the traditional SSD drives. Oh, maybe he didn't due to a slight dent to one of their other products.

[@PaulMansfield]

And people criticise linux for needing the use of the command line occasionally?

[@JuryDutySummons]

Windows in the server environment is really embracing the command line. They have put a ton of effort into making PowerShell extremely versatile.

[@The_Keeper]

Occasionally!?! Linux is basically a Dolled-up Command line interface.

[@davidjameswales]

@@The_Keeper Linux is a kernel. The Linux desktop is far nicer to use than the windows desktop is these days. Give it a go sometime. And if you are working with people who don't know how to type words to do things it's time to quit IT. Even webmin can't save stupidity.

[@reshadegaming6285]

Paul Mansfield This is so different though, I think those people mean for a day to day consumer.

[@jaizon]

@@N0N0111 If you really use linux you know that's true, most things are done in GUI now, even easier than in windows for sure

[@mareck6946]

@LTT Actually you can do it with ZFS ( and specify what you wanted ) you can even do it using freenas. However this will require some pre-planning of the Array then specifying enough fast ZIL / l2ARC caches - and giving the System enough Memory ( 64 GB+ ).

[@ManzaEN]

*I use a dell server as a computer lmao*

[@jyro1072]

I use a lenovo laptop Kappa

[@blatantpotato1367]

I'm sorry, I can't hear you with the noise of my Mac Xserve G5 dual CPU server's fans being at their lowest settings.

[@arwlyx]

I feel bad for your ears

[@yuyiboy]

You don’t know how to quiet it yet?!! 🤷🏽‍♂️🤦🏽‍♂️

[@Vaati]

you can get rack servers for less than 300$.

[@Ecker00]

Why are the editors not working with low resolution proxies, and only render full resolution when completed? Dont see pixel precision being very important.

[@ExtinctInsanity]

I still think it's funny how Apple came out with the "fusion drive" 3 years after hybrid drives... they're always late and claiming it's their idea. Lol

[@Raja-ev1ly]

This guy is crazy, he uses Windows Server

[@mpitogo]

I've used StoreMi in two systems, M.2 NVMe and large HDD. Seems to work great.

[@Stampmaster55]

I am a professional storage engineer and I can give you some pointers if you'd like. But generally HSM isn't usually as instant as your explanation. HSM usually works on a heat system, and data is only elevated to higher tier after a data block is hit several times, and then "cools" and is demoted when it hasn't been hit for a while. Your test would not have triggered normal HSM tier elevation. That being said, I can't speak directly to your solution as I've never used it, but I would guess that it is not really different. Unfortunately for your needs you'd need SSD, or at least 10K SAS disks that can generate sufficient IO with low latency. Most HSM configs wouldn't do video editing much good so the speed has to be baked in.

[@Stampmaster55]

I'd also be remiss if I didn't point out that consumer 10K SAS disks really aren't worth it anymore as they still under-perform compared to SSD and have essentially the same $/gb price point. For the most part, 40ms latency is the threshold you don't want to exceed, since most DB related tasked get really finicky with disk latency higher than that (and block level tech like FC and iSCSI start dropping connection)... here is a rule of thumb for the IOPS a disk type can manage before bad things start happening: SATA: 40 IOPS 10K SAS: 150 IOPS (can go a bit higher, but playing it safe) 15K SAS : 200 IOPS (these don't really exist anymore since the precision required in a 15K disk greatly reduced the maximum size potential... as well as made a lot of noise and heat and the power to maintain 15,000 RPMS is considerable) SSD: This varies a LOT, with consumer disks ranging up to 30K IOPS while the enterprise 16 TB (err.. 15.3 TB) SSDs from Samsung churn out 230k IOPS per disk combined read/write Assuming you want to stick with your existing hardware and minimize cost, I don't see how you can avoid buying more SSDs. Look on the bright side: SSDs make almost no noise, use a fraction of the power of spinning disks, produce less heat, are much faster.. all for a small up-cost when compared to SAS disks. Also, you could try to land one of those bitchin' Samsung disks as a demo... but they cost more than that Intel chip you dropped... so be careful! :)

[@Johnnyb167]

Linus "What if you could combine them into one?" Me "That's called a hybrid drive Linus."

[@EdmasterMM]

LINUS HAS HIS MOUTH SHUT IN THIS THIMBNAIL!

[@legin3753]

He didnt have his soy boy latte

[@SpookyLurker]

He didn't wanna earn money that way for this project.. 🙊🙉🙈

[@OnlyNotes]

LTT: Funnels money and resources into improving 8k video editing for youtube. Me: watches in 480p

[@DarkYuan]

I wish Canada would invest more in rural high speed infrastructure

[@OnlyNotes]

​@@DarkYuan I only lower the quality to make the most of my 80gb monthly bandwidth limit. I couldn't imagine having to wait for video to buffer. Makes me appreciate the speeds I do get

[@RandomUserName92840]

You know your server room is legit when you grab a jacket and a hoodie @8:50

[@chaos.corner]

Unless you grab a tank top and Bermuda shorts cause you're working in the hot aisle.

[@Impractical_Engineer]

That means your server isnt working hard enough and your over paying for cooling

[@chaos.corner]

@@Impractical_Engineer In a modern server room, the servers don't heat the cold side. The air is drawn through them where it gets hot and is then drawn away. How it's treated after that will depend on implementation.

[@permissionBRICK]

OMG Its my favourite show: Server-Crap with Linus!

[@hansypants]

"Only 24Tb of space"

[@proehm]

Place I used to work had 60T. It wasn't enough. Had to reach into LTO-5 storage quite a bit.

[@karanvora2674]

That's just a few weeks or even few days of storage for people like linus or a data scientist

That's 3 TB, less than a half of what I have in my PC.

[@kindanyume]

only 24TB of SSD space!!!!!!!!!!!!!!!!!!!! 24tb of oldskrool sfrisbees is easy and relativly cheap... the highend SSDs hes workin with are bat shit pricy as hell

[@beedslolkuntus2070]

Paul Roehm Really? Wow

[@HoshPak]

I run a 2x 3TB HDD mirror which is cached by 1x 1TB NVMe. I am combining md-raid and LVM on Linux in order to do that. The setup allows me to slice up the NVMe and apply it as cache to my virtual machines (and mounts) freely.

[@stephenreaves3205]

Says FreeNas/ZFS won't work because he can't fit enough hardware to cache everything, decides to use windoze and just not cache everything

[@TheShadowblast123]

That's actually a perfectly fine path of reasoning. If I'm going to an unfamiliar environment, it better work and he knows it can't, so why not try not caching and using windows instead?

[@afdasdfadga]

He clearly explained why he didn't use ZFS...

[@gordan79]

@@afdasdfadga no he didn't. It sounds like he just can't be bothered to evolve to something he can't control by clicking on pictures. Now, that's fine, but it's disingenuous to claim that ZFS won't do what his original stated requirements were.

[@brianewell2566]

Hey Linus, google "ZFS Prefetch"

[@joperez1980]

Linus, I'm a big fan of your videos, and I'd be very happy to help you. I'm in charge of a postproduction studio so this tips may work for your team. Just my 2 cents. What most of the high end productions do everywhere is that after shooting, we transcode all the footage to a format which requires less space, less cpu and less network bandwith. Generally, you can transcode all your footage into ProRes Proxy or ProRes LT so your editors will have all the footage available in their project with instant access. They can even edit on a mediocre laptop if you will, and it's useful to be able to edit on the go sometimes. After the final cut is approved, the project is then sent to the online editor, where the edit is relinked (conformed) to the high res footage and only the footage used in the cut gets copied to the 'online storage'. You can do this on DaVinci Resolve, where you can do color grading, audio mixing (or have it done on a ProTools, it's really up to your team) and do mastering and final delivery. I don't get the point of doing 8K videos for youtube, but that's really up to you. I think your videos are watched because of their content and they're very well produced. HD, 4K or 8K doesn't really make a difference. Moreso, your lighting and audio does and they're done really well.

[@Nj1498]

Complains about 7k for storage, but dismantles 50k camera.

[@Bassquake76]

Why do they edit stuff in 8k? Why dont they use low rez proxies then export at 8k when done like pro editors do??

[@TechnicalGamingChannel]

Bassquake because they’d still have to transcode the footage to do that, which requires fast storage and lots of local processing power

[@joebob2311productions]

Why not just edit in 4k to begin with? Most people dont have 8k monitors or the bandwidth and pc speed to run 8k

[@prime201]

@@joebob2311productions They explained this in previous video, it has to do with their editing process

[@JersenMan]

@@joebob2311productions I think brandon talked about this once, its basically to give the editor more felxibility to do little zooms (without loss of quality) or to (re-)center the frame for example

[@joebob2311productions]

@@JersenMan i know about this but if they want to save money it seems a decent option

[@Nite85]

watching the painful powershell stuff just gives me PTSD. from my every day life.

[@csharpcoffee]

I feel you, I find Powershell awful. I probably find it awful because I used Linux terminal 100x more.

[@b2bb]

Why not just run all this in a Linux server? Tiering and availability is far easier and more manageable.

[@getsouled]

If you need a fast SMB / Cifs share, Windows is your way to go. For NFS probably Linux.

[@Dcrev]

i imagine it would have to be due to the editors being familiar with windows and windows work stations and linux servers being incompatible? (im not familiar with linux so forgive me) it would slow down production of videos and other projects to make everyone relearn the OS

[@kalazakan]

​@@Dcrev The OP means changing the *server* from Windows to Linux tho -- not the Editor's PC's.

[@b2bb]

@@Dcrev Maybe it could just be a temporary growing pain that would eventually be more beneficial. Hard call.

[@fgwaller]

@@Dcrev There are ready to go storage distributions like FreeNAS, XigmaNAS (fka NAS4free) that do most of the interesting stuff for you and offer you a lot more options. With limited time to invest. I would however say a proper storage specialist would pay of for Linus even in the short run!

[@hblaub]

With Squarespace you can make your own beautiful storage server ;-)

[@tomoyat1]

The cache replacing algorithm Linus was explaining around 11:20 is pretty much exactly what Adaptive Replacement Cache (ARC) in ZFS does...

[@mrED123]

Now I'm happy to watch a 10 minute LTT video :)

[@tripdishman]

zfs is built for this. and no powershell required.

[@boulderdennis]

Question. If his video editing software is based on Microsoft technology is he still able to use zfs to manage his storage?

[@tripdishman]

@@boulderdennis 100%. The only hiccup is I believe his server is Windows server, which won't have great zfs support and will need to be Linux based or even better for zfs be freebsd, which is what freenas is built on.

[@boulderdennis]

@@tripdishman Thanks for your reply.

[@tripdishman]

@romaneeconti02 romaneeconti02 powershell was an attempt at copying those "archaic" shells so not sure how that's a winning argument for it. I agree that's what storage spaces was designed for but you're going to pay the Microsoft tax for running their server, and apparently have headaches along the way according to the video. ZFS is also built exactly for this, is free and open source and can run on a wider variety of hardware than Windows server or storage spaces. This video is describing needing more TBs of storage, specifically fast storage, so being able to define a pool of HDD as main storage, nvme drives as cache and still having arc in RAM is super useful. Again, all on free software which is another big point of this video, saving money. I agree both will work and knowledge of at least one shell is necessary, but one is a free solution that actually let's you do what you want and the other costs money doesn't seem to have any benefit over the other.

[@kofteistkofte]

​@romaneeconti02 Did RMS stole your bike when you were a child or someting? I saw your uninformed comments in a lot of post that suggest Linus to do something that BUILD for what he needs. I'm sorry but your dellusions about Windows being a more reliable and stable platform for servers are wrong. That's why nearly none of the huge services uses it. Cause *nix alternatives are better in this scenarios. And also the reason why open source software is popular on IT is security and flexibility. Yes you HEAR more zero day bugs, because you need to. Every software gonna have those. But good thing about everyone knowing about it is everyone can took steps to prevent it and have infinity more people to fix the problem, which is the reason people don't trust close sourced services. As long as the bug have super high risky stuff and can be prevented by user end, companies like Microsoft won't tell anything about it to the public and try to fix in internally. It can be done in a week or 10 years. Please get some experience beside MS ecosystem, it would help you in future.

[@ELMolliez]

Linus just out of curiosity, when you started this KZbin channel did you expect to become this huge of a media group? or did you just went along with it, like did it ever crossed your mind, you would have a server room, going to the heights of speaking how many Terabytes you have to allocate to just one project? I'm a tech enthusiast and i have been following you since the making your own pc tower for gaming tutorials, and never realized how much you as a KZbin-persona and as a media group have grown, there is even meme's i share with friends about you guys. I usually don't comment a lot but, great job man. I hope you never cease to produce content. love you all

[@jonmayer]

The real issue might be editing 8K footage for KZbin....

[@hellsgate09]

Not really, pretty sure they've at least mentioned, if not documented, that working with such high-quality files, when uploading them to KZbin, KZbin re-encodes the video in line with its own policies and results in a marked increase in video quality say at 1080p versus uploading a 1080p native source video(just one example) at the same 1080p on KZbin.

[@bladactania]

They don't only publish to KZbin

[@jamesmcgee1484]

So true, then its down sampled anyways, id like to see a comparison between a 1080 and 4320 down sampled. My guess is the difference would be minor if even noticeable after youtube re encodes it.

[@sablanex]

@@hellsgate09 I'd say the flexibility is more important than the potential increase in bitrate after youtube compresses it. Like if you have 8K footage you can zoom/crop do whatever while sill maintaining 4k quality. While if you recorded at 4k as soon as you crop or do anything like that, you'd lose quality.

[@vallerieknight198]

Back when KZbin rolled out 720p, they reencoded old videos which were uploaded in that quality. To check this out, look at the channel Captain Disillusion and look at how early his HD content starts (well before HD was a thing on KZbin)

[@deltasource56]

complaining about editing 8k videos while I'm watch this in 480p ...

[@lunchie80]

I'm only using 480p due to bandwidth limitations inside a 3 foot thick concrete blast resistant building. Doesn't buffer as often through the cell repeater. It's nice and quiet in here though haha.

[@ahightechlowlife]

Seriously, this, Linus. Nobody cares.

[@naza5759]

Weak

[@alexblack8634]

So how is life in prison?@@lunchie80

[@olinwread137]

Linus can come across as smart reviewing a video card or talking about a smart phone, but in the server room he just doesn't really do his homework nor consult real experts. I wish he put more money/effort into these videos about server room technology. Tiered storage is doable on a budget with the right solution.

[@CoolJosh3k]

“How did no one at MS notice this?” You do realise MicroSoft is part of the question? They probably notice, but fail to care.

[@silvy7394]

Hey, Linux fanboy. You forgot to put $ instead of that caps S. Thank me later.

[@CoolJosh3k]

Sapphire Destiny I use Windows, but my good god there are plenty of bugs.

[@silvy7394]

@@CoolJosh3k Get out of the Windows insider program, then.

[@CoolJosh3k]

Sapphire Destiny I’d not want to deal with even more bugs from the beta stuff. For example, if I try to safe remove an idle external drive, my whole system effectively locks up and Windows won’t finish a shutdown.

[@silvy7394]

@@CoolJosh3k And why are you removing drives while the OS is shutting down? lmao. No wonder why you're having issues. No common sense at all. Let me guess. You take the battery out of your car and cry about how it doesn't finish starting up? *Aware that wasnt a proper sentence*

[@WyattCieluch]

I really enjoyed this video, I'm going to University for this sort of work and this gives gave me a good laugh. Not a day goes by that I don't wanna tear my hair out while working on Windows server

[@justinstewart950]

Linux/unix >>>

[@costafilh0]

I don't understand why you need this! Wouldn't it be cheaper/easier to divide the storage in multiple servers?

[@lo_rez26]

"A Better Way" might be to consult with a professional who specializes in networked data storage for video editing instead of kludging something together.

[@JayVal90]

...but would that be an entertaining video or remind me of myself attempting to pull this off?

[@worldrecordegg1238]

When you drop everything and blame the manufacturer for high price: P.S. It's just a joke. We all love you Linus.

[@imwithyou38]

honestly these kind of videos are why i like LTT, it shows more then just all the successes

[@saveraptor2209]

"Work the way you'd normally work" *opens facebook*

[@alidan]

thats why their pay is shit.

[@taski1]

*"opens youtube"* fixed that for ya

[@MaxAndreZils]

That fan in the background is trying its absolute best to cool an entire server room.

[@HardwareNumb3rs]

What you should try (as a cheap solution) is an array of 8 or more sata drives configured in raid 5 in a QNAP 8/12 bay NAS, for video editing you need spindles, having nvme or ssd is overkill, all that iops are for VMs or DBs. Or well, in alternative there’s always a Dell/EMC entry level enterprise storage, but maybe you don’t have that need yet. Out of curiosity, what are you using as backup? (Cloud archiving?)

[@Xiph1980]

Cough*ZFS*Cough.... And no, it's not difficult. It's actually rather easy.

[@Taikamuna]

This is the definition of first world problem

[@5iwot5]

The editors are really of a different breed. Fascinating to look at!

[@joeyhillers9460]

Get a raspberry pi. A little setup and a few adapters is a lot cheaper

[@Bdmaurice]

That's a great idea! Anyone have any objections? Usually when a good idea comes up it's shot down by some cynic.

[@exit7142]

@@Bdmaurice yeah, but you should make a cluster of two raspberrys for 8k editing. one is probably not enough

[@Lisedah]

The bandwidth of the ethernet adapter is shared with the USB, you wouldn't get great performance if you use something plugged into ethernet and USB at the same time.

[@nonameentered1918]

cluster 3 of them then! woah!

[@springbok4015]

And a whole ton slower