How Do Databases Store Tables on Disk? Explained both SSD & HDD

in this video I explain how database systems store their data on Disk both f from SSD (solid-state drives) and HDD (HarD disk drive).
There are many factors that go into that discussion so I tried to cover the high level here.
Reseources
www.extremetech.com/extreme/2...
www.percona.com/blog/2019/12/...
🏭 Software Architecture Videos
• Software Architecture
💾 Database Engineering Videos
• Database Engineering
🛰 Network Engineering Videos
• Network Engineering
🏰 Load Balancing and Proxies Videos
• Proxies
🐘 Postgres Videos
• PostgresSQL
🚢Docker
• Docker
🧮 Programming Pattern Videos
• Programming Patterns
🛡 Web Security Videos
• Web Security
🦠 HTTP Videos
• HTTP
🐍 Python Videos
• Python by Example
🔆 Javascript Videos
• Javascript by Example
👾Discord Server / discord
Support me on PayPal
bit.ly/33ENps4
Become a Patreon
/ hnasr
Stay Awesome,
Hussein

[@yashbudukh340]

Amazing content man.Please make a video on sql vs no-sql and when to choose which.

[@timothyruszala4973]

What an amazing speaker-You're most engaging speaker on software engineering I've ever seen!

[@abhirishi7]

I mostly watch your videos to hear your accent..its super cool. and besides you provide some really good information too.

[@hemantsinghjadon849]

Really informative. Would love to see some examples of this actually affecting the engineering decisions companies made.

[@shubhamgupta-to7so]

Thank you so much Hussein Very Informative. Never knew this before. Btw the sound effect at 02:52 was exactly the same when I do I/O stuffs in my system. hahahaha

[@sundaramjha1776]

Thank You for making this video. I have requested for it. Keep sharing ur knowledge with us.

[@hnasr]

Sundaram Jha yes it was great suggestion! Video is getting popular.

[@harshgandhi4328]

Awesome content Hussein. Thanks a lot. Can you make a video as to how memory management works in NodeJs/Chrome V8?

[@davidraj2862]

Amazing absolutely love your content 👌🏻

[@nekoill]

My man, I gotta say I love the way you present information. I'd love to be your neighbor and spend time just rambling on about curious IT stuff.

[@8Trails50]

This channel is a goldmine

[@jondoe79]

Superb content as always 👌

[@dudibs1]

great intuitive explanation. thx

[@AbleToLiveHere]

Great video. As usual. Thank you.

[@hnasr]

My pleasure!

[@techwithameer]

Awesome infos bro

[@ashutoshmishra2328]

Hey Hussein, thanks for this video 😃 Can you make a video on clustered and Nonclustered indexes also, it has a major role in storing records of tables in memory and it is bit complex. I tried to understand it but didn't understand much. It'll be great if you can make a video about it. Thanks 😃

[@hnasr]

Great topic

[@amitmahadik4653]

Really great explanation, just curious to know how cloud storage stores data

[@9831599481]

Ur videos are dope

[@AnantaAkash.Podder]

Wow... Amazing content sir...😃

[@suryabhusal1527]

Hello sir, i really enjoy your videos. Those content are rerely found while entering into microservice jungle :P. But i've one issue. I've watched your all backend engineering playlist. But i'm still confused on how to implement Saga Orchesteration using RabbitMQ. Please Please i need this!! Thank you for such wonderful videos. Love Love !!!!!

[@MuztabaHasanat]

It's a damn good video!

[@pokiripandoo]

Respect for the PS on the back ❤️

[@janetachieng8990]

very nice,got it

[@timbui5556]

Hi Hussein, how can I put my Postgres data on a portable SSD so that I can use the Postgres on my desktop and laptop without restoring? Thank you so much for your teaching!

[@fatememalekan3522]

that was perfect

[@RahulSharma_rsconcept]

Amazing content! :) Just to add to the last mention on Rocksdb, from what I have read. RocksDB is a fork of LevelDB, which itself is an open source storage engine implementation off from Google Bigtable’s tablet storage engine (proprietary ). But yeah, each of these system’s underlying data structure is an LSM Tree. They give you higher write throughput vs BTrees which performs better for reads.

[@hnasr]

Rahul Sharma thanks Rahul For the additional info! Correct! RocksDB is a fork of Level I mentioned that on my database engine video Database Engines Crash Course (MyISAM, Aria, InnoDB, XtraDB, LevelDB & RocksDB) kzbin.info/www/bejne/gWq0lWaLgq2Ih5Y

[@saurabhsharma7123]

Keep up the novel work. Hats-off!

[@rahulspoudel]

Keep up the great work man. 2:51 LMAO

[@euriskoo]

Hello Hussein, why do we need to the virtual memory when we retrieve a data from a disk?

[@5_inchc594]

"It reads the hold god damn block ." hhhh your video is funny, thanks for the explaination .

[@section9999]

Very useful stuff. 👍 But I have a question! Would you mind sharing the link @15.42? I was thinking of building a project for DB algorithm visualization so I want to make sure its both distinguishable from that and useful in a practical sense for the average DB student and professional.

[@hnasr]

Sure thing! www.cs.usfca.edu/~galles/visualization/BTree.html

[@section9999]

@@hnasr Cool man i greatly appreciate that!

[@singaravelann3671]

Thanks for this video, Can you explain why RAM is faster than disk seek and can you explain how the data are organized in ram.

[@allanguwatudde7623]

Ask ChatGPT

[@arjungambhir94]

Lovely, great content ❤️, can I know some of the names of books behind you? Looks so interesting

[@hnasr]

Thanks 🙏 Sure I made a video kzbin.info/www/bejne/a2q9ZHeiiNprqrc

[@jackedelic9188]

so does it mean given a choice, i better use LSM-indexing db engine (cus btree rebalancing killing ssd)? How easy/hard isit to switch db engine?

[@lucavogels]

How do Key-Value stores actually store the data on this? Because size per entry isn’t that fixed as in relational DBs

[@shobhitmittal77]

Awesome Content Sir.. Reminded of The Joker in Dark Knight!!

[@youtubewatcher5500]

How the database programm can find the beginning of the block on the disc?

[@snake1625b]

Is the extra speed of the SSD worth the extra cost?

[@420_gunna]

Yeeesssss Hussein. I don't remember if i recommended this, but you have _GOT_ to take Andy Pavlo's database systems and advanced database systems courses on KZbin from CMU. Best class I've taken, covers a lot of things (esp the advanced course) that I think you, me, and a lot of people here find interesting. Enjoy!

[@420_gunna]

First lectures of each here: Intro: kzbin.info/www/bejne/pZa8c5edncZ8oMU Advanced: kzbin.info/www/bejne/iZW6ZoWBiseigcU

[@420_gunna]

Also, the MIT 6.824 Distributed Systems course on KZbin. Taught by Robert Morris, the inventor of the first computer worm! (Also the UWaterloo and UCSC Distributed systems courses aren't bad ;) )

[@420_gunna]

Oh by the way designing data intensive systems by Kleppman and Database internals by Petrov are must reads! :)

[@hnasr]

Sam Silver Awesome ! Thanks Sam for sharing

[@apusingh1967]

yo' grt Nasser lv u there is some confusion HDD sector size is mostly 512 BYTE, and OS reads in sectors nothing less.. so OS blocks are same as sector looks like. SSD doesn't have sectors. It has pages and blocks. The block size is mostly 512 KILO BYTE! Pages are usually 4 KILO BYTE. So one block has 128 pages in SSD. OS running in native mode will read/write in pages looks like. But garbage collection etc will work with blocks, since updates are rather intensive as you mentioned.

[@lieblius]

Great video explaining things at the OS level, but aren't databases interfacing with file system api's to abstract this all away? I don't really understand the reason to bring databases into the conversation here. Edit: I see this is part of a database engineering playlist. I think it would have been clear to me if the OS level was mentioned a bit instead of jumping straight from database to what happens on disk. That made me think for a second that database implementations were mainly going straight to I/O instead of using the native OS tooling. I was looking for explanations on how databases handle concurrency when writing to files as well as explanations on the format in which the files are stored.

[@AAZinvicto]

People, If you want to do a deep dive into Databases, search for "CMU Database Group" on youtube. Great video Hussein! Keep it up!

[@hnasr]

Thanks !! Yes Sam shared the resources those are excellent. 😊

[@hnasr]

Thanks !! Yes Sam shared the resources those are excellent. 😊

[@AAZinvicto]

@@hnasr Oh, didn't notice. :D

[@montassar_akrmi]

Are you really appreciate the tech behind it? What does "ugly disk" even mean?

[@abhishekhamal3126]

How can I learn more on what you are talking about?

[@sameerakhatoon9508]

text books

[@bhys570]

Can someone explain to me why he said the data is stored on four bytes

[@prashanthkumar0]

This video is really scary , much scary than watching insidious , Reason: Just upgraded my hdd to ssd and going to learn dbms by making one ( using B-Trees ).

[@jameskennedy674]

Nice video. However all the sound effects do not add anything to the content, or provide comedic effect.

[@thinkingcitizen]

had to hit pause within the first few minutes to imagine a beautiful table.... 10 mins later, still nothing...

[@aman2426]

How many accents...italian, french, german, indian, arabic..

[@miresoman1769]

Sorry bro but 'sasta' Zayn Malik

[@vaibhavkumargautam]

Bkl sahi se padhna sheekh.

[@dancingdev1088]

Why does he act like Neil degrasse tyson?