#03 - Database Storage: Files & Pages ✸ Neon Database Talk (CMU Intro to Database Systems)

[@americanjetset5010]

As a long-time data engineer, who has been using DBMS my whole career: THANK YOU for posting these online for free. So much info.

[@_4_632]

19:50 Disk Oriented DBMS 24:40 File Storage 27:40 Storage Manager 30:02 Database Pages 42:57 Page Storage Architecture 44:02 Heap File 46:11 Heap File : Page Directory 49:05 postgres pg_freespace 50:50 pg_relation_filepath 59:47 Page Header 1:01:15 Page Layout 1:04:19 Slotted Pages (header keeps track of used slots, the offset of the starting location of the last slot used) 1:08:57 Neon - Serverless Postgres

[@Rohinthas]

Neon is such a cool idea... also really glad I got this overview of pages, it makes a lot of sense and is (conceptually) way easier to understand than I assumed

[@evanxg852000]

[1:00:21] Some background talks he might not be aware how sensitive the mic is this semester :)

[@robwells3713]

I think it's from the guys from Neon who join remotely to give the lightning talk.

[@evanxg852000]

@@robwells3713 That's what I thought when they switched to their presentation ☺️

[@daver94]

Amazing! Neon is quite impresive :) Thanks

[@tranminhhaifet]

thank you! such as great details into databases

[@Lecker9419]

41:20 Hey Andy, let's say I made changes to a 16KB page , but my changes are super small , only a couple of bytes, why is it necessary to write the whole 16KB page back to disk ? why can't I write a small chunk e.g. like a hardware page size of 4KB out of this 16KB page back to disk ? e.g. 16KB page is made up out of 4*4KB hardware pages, maybe one could track which one of these has been modified and lock the whole 16KB page before we only flush that 4KB chunk(s) to disk instead of the full 16KB page size. Or is it too much hassle ?

[@andypavlo]

You are assuming that the page is exactly the same after an update. The slot array could change (first 4KB) and then the change could be farther in the page (e.g., third 4KB). It's just not worth it to try to be clever like this. But this is what an LSM solves for us (see Lecture #4)

[@slabchan7310]

damn, Andy is super cool

[@SteveLoughran]

Real shame that octane didn’t survive. We developers need to look at how we failed to take advantage of the possibility

[@hazembenabdelhafidh3286]

Hey, thanks for the upload. Really helpful, but can we get to know what are the projects you're giving to the students it would be great if we can do the projects as well

[@andypavlo]

All projects + homework (with solutions) are available to anyone outside of CMU. See course website. -- Andy

[@mohamedmirghani1747]

The notes link is invalid, could you please fix it

[@UlyssesBaldwin-y2o]

Katrina Glens