How to Achieve a Petabyte of Storage

R&D Engineer Brett Kelly gives an overview of how storage users can achieve a petabyte of storage for under $100,000. In this overview, Brett showcases all the pieces needed to create an affordable storage cluster using the Storinator Storage Servers and GlusterFS.
For more information, check out www.45drives.com/products/cluster

[@shiiiguthewise6778]

Finally, now I can store all of my hentai!

[@evanwilson410]

White Rice™ almost all but not quite

[@darkarmaeffect3738]

fax xD

Whumch, download now 8k hentai.

[@darkarmaeffect3738]

@ fax xD

Or store all Flight Simulator 2020 files on it. Would take arround 100 days to fill with my gigabit FTTH connection.

[@learnalanguagewithleslie]

Give it 10 or 15 years or so, you'll be able to fit that 1Pb onto something the size of a microSD card, LOL. My first 3.5" HDD was like 20Mb only (not the cache, the whole thing) hahaha.

[@yoursleepparalysisdemon1828]

I'm thinking that'll be a terabyte in a few years (they have them, but aren't very cost effective). it depends on the transistor reaching the size of an atom.

[@seesharpist]

2.5 years later and you can get 10tb golds for 370$ instead of 8TB for 550$ That cost goes way down

[@lindseybauer9097]

What can't you do with a gluster cluster of 1.1 petabytes? Time travel?

[@pithagorians231]

Hello, how does glustefs behave when constantly reading and writing many small files?

[@45Drives]

Hi Vitalie Brinza, GlusterFS does not work as well with small files in our experience because it does not use meta data. Therefore, it has a hard time “looking” for small files. A small file is considered to be a file that takes longer to look up than to actually do an operation on it. Thanks.

[@pithagorians231]

Im asking because about 3 years ago i used to have a glisterfs cluster with 2 nodes with less then 1 TB data used for storing uploaded images and cache files in a web commerce site and i had issues with volume consistency. Im wondering if they fixed such issues until now as i plan to implement a multi PB storage solution and looking for a technology. So far only Ceph seams to worth trying.

[@Mekugii]

YOU have 2.5 petabytes of storage in your BRAIN

[@luckydavis2157]

what do you you use to hold the things i have no idea.

[@45Drives]

I'm not exactly sure if I understand your question, but we use our own storage server called The Storinator to host open source cluster software. To learn more about The Storinator, clustering and our other storage solutions, please check out our website www.45drives.com/products/cluster/ or blogs!

[@dangi12012]

I think its not cost effective to buy HDD space for 2x the price of a normal disk "gold standard" as he calls it. If 1 or more drives fail there wont be any data loss anyway if you replace it soon. To put in perspective: normal HDDs cost 25 dollars per TB. which means the whole Petabyte will not cost 83k but 30k including spare drives and the server itself.

[@sadeqalbana]

I'm using WD Purple 3TB models that are designed to work 24/7 and they cost 100$ so it's 33.33$ per TB The funny thing though is the 4TB Model of this drive cost about 165$ I think these enterprise drives are a waste of money even for enterprises

[@s_steadman]

Imagine one Petabyte of 970 SSDs

[@mdd1963]

4 years later-- 31 TB NVME SSDs announced! :)

[@oursours]

Who would spent so few money on the server hardware, when you spend that much on the harddrives ?

[@45Drives]

Hi Oursours, I’m glad you brought this up! One of the things we are super proud of here at 45 Drives is the ability to sell an enterprise-grade, very dense, and scalable storage servers to not only the larger companies but also the small - medium businesses as well. To keep our price low our HDD’s are directly wired to our 10GbE NIC cards and we use enterprise class off the shelf components that other companies would charge you much more for. We are in the business of providing storage solutions to all levels of business and making them affordable is what we strive to do, we just don’t see the need of charging our customers more than needs to be charged.

[@mdd1963]

@@45Drives " To keep our price low our HDD’s are directly wired to our 10GbE NIC cards " Not entirely sure what you actually *meant* to say here...

[@firelord7776]

Achieve

[@llothar68]

I can get the drives for $25000 where is the rest of the $60000 going? To Brett Kellys overpriced company products! The CPU and RAM of all three machines is worth less then $1000.

[@manw3bttcks]

The video is from 2016, so your 2018 drive prices are a bit lower and sizes bigger. These are enterprise rated drives, I suspect you're price el cheapo commercial ones. Also with 45drives you're getting a warrantee, recovery services, support. It all cost money. With you rolling your own servers you get none of that.

[@SalvatorePellitteri]

Gluster is old way to do storage( raid over the network). As I said an old way to build storage, for more than 100TB you should use Ceph that is very modern distributed object storage system.

[@blakebutler4956]

ayy LinusTechTips just got some of these

[@RAMtrails]

and they don't even have a real petabyte

[@j7a1k1e]

Zimba's Doom Corner what do you mean? they do have a petabyte.

[@blakebutler4956]

j7a1k1e yea, but it doesn't quite come up to a petabyte. good enough imo

[@j7a1k1e]

That's like being upset that you don't get exactly 1Tb on your 1Tb drive. It's never exact. And since there are so many drives, the loss adds up

[@xcruell]

Its close enough tho. You can still count it as a petabyte.

[@MsUltrafox]

PRICE UPDATE The Storinator has gone up in price with $100,- for the base 45 BUT the hard drives have gone down in price a LOT. They were $540,- each but now they are only $325,- each So the price of $82.550,- comes back down to 54.237,- ($54.544,- with shipping) 28 THOUSAND dollars less. Holy stinkballs.

[@kissmyterminal4557]

@Habituated Abnormality It would now cost you 23500 to get a petabyte of storage if you bought internal hdd from amazon each being 8tb

[@corcon6976]

@@kissmyterminal4557 Annnddddd.... that is a cost per every ~3-5 years due to limited hdd life. Which means you need to bring in at least that much in that time to recover costs, plus electric.

[@SalmonellaStew]

How bout now?

[@shanky.y]

I'm just gonna wait 20 years and all this big stuff going to be like SD card for 20$ not 90k 😂

[@excrafter7419]

in 20 years youll be able to get a 1 petabyte hard drive for the same price as 1 terabyte right now at least according to moores law

[@pgcharlie]

:))

[@807pranavghandade8]

Not to mention the fact that in 20years due inflation 20$ would be way insignificant.

[@andrewyork3869]

@@excrafter7419 mores law that is rapidly ending....

[@jeffwads6158]

You will be in the Matrix by that time so don't bother. Dibs on Anna Nicole Smith!

[@PTNLemay]

It'll be amusing to see how quickly this video becomes outdated. Not to say anything about a lack of quality in it or anything, it's just that hard drive capacities are still growing at such a formidable pace. It might not be that long before we're saying "Oh a Petabyte? I got that upgrade in my tower for under 500$."

[@alessardarawi692]

PTNLemay damn lol a $500 petabyte I'm gonna die old before I see that

[@sparkplug1018]

Honestly, I'm 31 and didn't think id see a 10TB drive. So who knows what they will come up with in the next 20 years.

[@PTNLemay]

@VivaLaHinderaker This video is using 8TB units, so... while that is a step up, it's not a huge difference.

[@fizk]

We can already store 360TB in a small crystal disk: www.digitaltrends.com/cool-tech/data-storage-technique-packs-360-tb-data-glass-disk-eternity/ So it won't be long before we're all able to store a petabyte.

[@llothar68]

Well the last 6 years, a TB storage became more expensive then before. In the last 3 it was the same for SSD. So i dont expect anything from the next 10 years. 30 Euro per TB is the magic asymptotic line.

[@cooliteh]

I should probably pay my mortgage first. :)) Nice video.

[@TheShadows007]

2018: 1PB in 1U (32*32TB) with Intel "ruler" formfactor NAND flash-drives^^ lol

[@ygerber]

What I'm wondering is if you can make use of deduplication to get more out of the 1PB. We have NetApp Clusters which dont really cost more and with dedube we can also get to 1PB (nearly 18:1 ratio). And with NetApp we have great AutoSupport in case of a failure. Like a Disk Failure we automatically get a new disk. I wiuld really like to try one of these boxes and compare performance, capacity and overall price for 4-5 years in production.

[@bandaibrozendrinkswater3790]

Wow I am Really great to get a petabyte of space :)

[@blackgenesisishere]

now that 16 TB hard drives are coming out very soon i guess this same setup with 16 TB hard drives would yield 2.2 PB, or could achieve 1.1 PB in half the size of this.

[@cosmickatamari]

i'll take 2

[@mdd1963]

16 TB drives are now reasonably priced, jumping just $25-$30 for each 2 TB until You get to 18-20 TB units, at which prices then jump $70-80 for each additional 2 TB.

[@VibrGames]

Thanks for this. I had some spare cash lying around with nothing to spend it on.

[@maYdaY1337]

1.1PB of Raid 0 *thumbsup*

[@aaronroman2479]

Definitely will not purchase. If one drive fails my entire 1.1 Petabyte porn collection will be lost! SAD

[@aluandcache7336]

@@aaronroman2479 That's not a case on RAID 0

[@aaronroman2479]

@@aluandcache7336 I WAITED TWO YEARS TO FIND THIS OUT?!? Hanging myself tonight

[@aluandcache7336]

@@aaronroman2479 You should be happy that you found it at least

[@frank2398]

I love that 1 second after he says "for 24/7 uptime....." the video show gluster2 DOWN.

[@ghhggghcrjuhherg1895]

This is pretty incredible but what you gonna do with a petabyte of storage

[@shensnerl8004]

pron

[@Technikapfel]

If they weren't so expensive i would have already gotten a storinator for private use :/

[@kraio-sfu]

0/10, not enough “bud”s

[@henrikasromanauskas9237]

Good vid, but the guy seems to be high on something

[@theonewhowas7709]

only $83,000 LMFAO... get real

[@U_Lambda]

Drop in the bucket for enterprise businesses

[@adriancerv7312]

and it costs you less than one hundred grand...i'll buy it if you lend me a million.

[@gcasey2010]

lol Linus brought me here!

[@criticaltexan2334]

same!

[@anuradhatonge5811]

Me too !

[@tech-hilfeportal6611]

Me four

[@majstealth]

your mis-use of the totally good u´s disturbs me internaly... 1:15

[@shankarshanmukhopadhyay8372]

What's the largest cluster you guys have setup yet?

[@asdrubale3374]

Why a petabyte? To play COD modern warfare

[@charliebrownau]

Whats the temp like when you have that many DRIVES powered on ?

[@45Drives]

Take a look at our blog on thermal evaluation: www.45drives.com/community/articles/thermal-evaluation/index.php

[@nickchan6498]

No one petabyte is 1024 terabytes

[@blackturbine]

So is I want to make a brain (inorganic) I whould drag around a cabinet Makes you think about life (Pun)

[@xmurisfurderx]

these systems are incredibly overpriced, poorly designed and underperforming

[@mdd1963]

Compared to what?

[@piavon258]

at linustechtips you just look like a normal guy but damn here at 45drives you look a very smart guy hahaha, cool dude

[@cybernaticxe2928]

I can reach 3 petabyte with ssd

[@cosmoshuawei4659]

Is it under a 100 grand ? 😃

[@charliebrownau]

Do they do staggered powering on of the drives ?

[@45Drives]

The 2N redundant PSU is powerful enough start up to 60 enterprise drives (10TB) with no staggered spin up. With that said staggering spin up is recommended, as its nicer to your PSU.

[@MikeTrieu]

@@45Drives Whoa. Now that's some cold cranking amps! 🤯

[@musicunderground6324]

83 thousand f that lol 😆

[@RileyBanksWho]

I'm at 22 TB storage right now, one day of shooting takes up about 1 - 1.5 TB.. I never thought I was gonna have to invest in a 5 figure storage solution... sigh

[@JeffM375]

22tb... thats lightweight guy

[@shadowhunter388]

3 of these and you can map out the Human Brain. Congrats!

[@alexander92ck]

I don't quite understand , this solution uses 3 racks * 45 drives * 8 tb for a total of 1080tb, when they say "archive a petabyte" do they mean without any redundency and backup?

[@45Drives]

When we said 1080TB that was when we striped the volume, so there was no redundancy. Depending on your RAID configuration, you could still end up having a petabyte of storage. We would typically recommend something like a building 3 VDEVs with RAIDZ2 (RAID6), each having 15 drives in each chassis. This would mean you’d lose 16TB in redundancy per VDEV, so your storage would be 3*39*8TB= 936TB.

[@DanielSRosehill]

Amazing - thanks for sharing!

[@EVGUY13245]

It only costs 83 thousand dollars lmao

[@MrdotApple]

Hi! This storinator glusterfs can be mapped under vmware esxi? So I can use it like lenovo storage or its the same?

[@45Drives]

Hey Mr. Apple, You can map Glusterfs storage under VMware. The supported protocols are NFS and iSCSI.

[@Vic-fh9nl]

The biggest problem ..... how to use more than 1%

[@Solkre82]

R.I.P. CentOS

[@Hybridious]

Dude you can store tons of adult films.

[@mrjonnoma]

I can't believe you even think of using RADI0 on 45 drives at once... Just unbelievable... So single disk doing fancy stuff on just one of the 3 machines, would surely put one of those 3 machines down... Also if you use at least 2/3 of capacity in cluster FS (like, you want to survive that one-disk breakage and one of 3 machines going offline).. you actually will never get usable PB and whole hardware setup is just unreliable. What would be much better and production-ready is having 45 drives in at least 4X 11 DRIVE RAID6 volumes, plus one spare drive, that would give you, for 8TB drives, around 9X4X8TB=288TB physical space and one spare drive per box. That translates to 576TB physical space if using 3 boxes and 2/3 of capacity (to be able to survive at least one box going oflfine) with cluster FS, and logical space will get a bit lower then those 576TB. That would need ZFS on those machines and surely more RAM then 8GB to have at least some use of caching and not to mention deduplication in ZFS that can save you a ton of space IF you yave just enough ECC RAM in machines And supposedly an JBOD SAS on SAS or SATA on SATA controller WITHOUT expanders ...and would prove pretty much more solid than using RADI0 :) And yeah, you would need an OFFSITE Tape/offline backup capacity for those 500+TB of usable data space - don't play with fire without backups, folks! :)

[@Orkinman831]

It's not RAID 0. If you watched the video they're using ZFS.

[@mrjonnoma]

ZFS can also use RAID0, actually it does it by default, if you don't pay attention when creating zpool, it creates a stripe set. RAID5 equivalent is raidz, RAID6 is like raidz2, yet OpenZFS does it better then any hardware RAID and is multiplatform. (illumos,osx,zfsonlinux,freebsd) Problem with this video is that they don't mention you won't get usable space as said, but lower usable space if you use multiple RAIDZ(2,3) level Vdevs made from multiple disks, that zpool is made from, as I explained already. So it's not quite Petabyte and you really need multiple of boxes to survive hardware failures.

[@wolfdendrones8640]

You can get 6tb drives for 229 now means a petabyte of data would only cost 40k not 100k

[@j7a1k1e]

Wolf Den Marketing good luck finding an enclosure for all 169 of those hard drives

[@konsul2006]

I've got a 4TB drive and lost track of all my data and backups - It's just too much data to manage. You got 1000TB? I almost feel sorry for you guys :D

[@user-jt5vm3mi1w]

Use cloud, I have unlimited

[@CJWarlock]

Wow. Supercool! :) Congrats on the good product.

[@buck3401]

1000 1 tb micro sd clustered together

[@Eddyhartz]

what if you allocate 1 Petrabyte to Minecraft does it still lag?

[@mdd1963]

If you misspell Petabyte, then yes....

[@the_danksmith134]

I already found a way to create a single petabyte file! Just put your computer solve "π" 3.14...

[@nickchan6498]

or e=2.71828...

[@gamingscott7770]

Is this possible on mobile?

[@Akaizer100]

Very solid content from a server manufacture! Thanks for the Quality Upload, and thank you for making the internet a faster, larger place!

[@giovanniberry1611]

Less than a hundred grand

[@notorious8052]

Give me one of those

[@Maxt3r1]

more og this please:)

[@ansible9340]

this R&D dude is high on HDD

[@thesimulatorgamer3359]

But I can buy a house for 100 grand

[@thesimulatorgamer3359]

Атанас Паунов but still who need s this much storage unless u run a company

[@Haloheld]

Nice video :)

[@berend-janvandellen2497]

if you want to have a real petabyte, you need 1024 terrabyte, not 931, and each terrabyt should have 1024 gigabyte, not 931, and 1024 mb per gb, that would be a real petabyte, greetings from germany

[@Michael19830106]

Plain wrong.

[@DensDigitalDen]

Now this looks fantastic!

[@geogmz8277]

So Linus paid 83K? damn!

[@kazi1]

He got them free with sponsors.

[@Mina-zx6wh]

You really love pr0n, lol... 😂

[@beedslolkuntus2070]

Bad mindset

[@justyulik3697]

Wil it mine burstcoin ?

[@45Drives]

You can, but it won't work as well as other solutions. Mining requires a lot of GPUs, which are not normally included inside a Storinator. However, we like to develop custom builds and have a bit of experience in data mining. If you would like to further this conversation, reach out to us at [e-mail] !

[@stargazer1733]

45 Drives Except Burst coin isn't mined with GPUs, it is mined with hdd space.

[@Hi_Abadi]

wtf build your self will not cost that much gad

[@michaelrichter4941]

vidyaWolf no the guy is not that smart

[@Hi_Abadi]

who you mean me ? Michael Richter

[@michaelrichter4941]

Moli no no

[@Hi_Abadi]

Michael Richter thank you

[@SlothMotions]

As low as 83,000 dollers Ummmmm...........that is not low

[@mdd1963]

It is if you want a cluster stuffed full of 135 each drives at $500 a pop… ( prices 6 years ago)

[@lazerbrainzz]

lol 83,000.. One of these days I'll take you up on that. XD

[@mdd1963]

Check out licensing costs for MS SQL Server Enterprise in dual socket 64 core Epyc CPUs! :)

[@lazerbrainzz]

@@mdd1963 oh my goodness that is something to think about. Per core yeah?

[@THE16THPHANTOM]

annual licensing fee = zero i remember learning that there is catch to free/open source solutions. a catch that makes it so that the cost of maintaining(or something) of a "free/open source" solution outweighs the cost of an annual license.