Clustered Collections makes Mongo faster but there is a cost

Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon)
database.husseinnasser.com
In version 5.3, MongoDB introduced a feature called clustered collection which stores documents in the _id index as oppose to the hidden wiredTiger hidden index. This eliminates an entire b+tree seek for reads using the _id index and also removes the additional write to the hidden index speeding both reads and writes.
However like we know in software engineering, everything has a cost. This feature does come with a few. In this video I discuss the following
0:00 Intro
1:00 How Original Collections Work
6:50 How Clustered Collections Work
10:00 Benefits
17:00 Limitations
Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon)
backend.husseinnasser.com
Fundamentals of Networking for Effective Backends udemy course (link redirects to udemy with coupon)
network.husseinnasser.com
Follow me on Medium
/ membership
Introduction to NGINX (link redirects to udemy with coupon)
nginx.husseinnasser.com
Python on the Backend (link redirects to udemy with coupon)
python.husseinnasser.com
Become a Member on KZbin
/ @hnasr
Buy me a coffee if you liked this
www.buymeacoffee.com/hnasr
Arabic Software Engineering Channel
/ @husseinnasser
🔥 Members Only Content
• Members-only videos
🏭 Backend Engineering Videos in Order
backend.husseinnasser.com
💾 Database Engineering Videos
• Database Engineering
🎙️Listen to the Backend Engineering Podcast
husseinnasser.com/podcast
Gears and tools used on the Channel (affiliates)
🖼️ Slides and Thumbnail Design
Canva
partner.canva.com/c/2766475/6...
Stay Awesome,
Hussein

[@hnasr]

fundamentals of database engineering course database.husseinnasser.com

[@ke_mbo]

Day 1 of waiting for Hussein to make a video on consensus algorithms

[@hnasr]

i tried to read into them few months ago and haven’t picked up the pace.

[@joshcho96]

Thank you so much for your insight everytime :) I am learning so much from your videos.

[@mohammedabdulbary1577]

another amazing video, love you man ❤

[@Ghost_1823]

We are heavily using clustered index in our app. But one drawback was use of UUID and creating own clustered index. Thanks this video helped to avoid bottleneck

[@husreason]

Can we please get a video on secrets management? Love the breadth of topics you have covered on your channel (thankk you so much!), but this topic seems to be missing, so I'd love to learn it from you!

[@EddyCaffrey]

Great video. It is a great addition to the database.

[@tesla1772]

Since b trees are aslo storede in files and pages. Do db fetched entire btree when an index scan/seek has to be done

[@adarshk7]

About the secondary index being preferred, I could imagine a composite index being more selective, where the > 2 IO would be less of a cost than the lost selectiveness. Maybe more so in range queries. So I guess it depends on your query in the end (where if you wanted custom behaviour you could even go for $hint). What do you think?

[@marsha363]

Awesome talk as always! Regarding 18:00, why would you want to do a query with the _id, and another filter, while the _id is unique? For kind of “is exist” query?

[@hnasr]

one example is a range query, give me all documents between id10 and 50 and having certain field is particular value , if that field is indexed it will be preferred over id

[@juliussakalys4684]

Whenever possible UUID strings should be converted to binary and stored as binary in the DB itself. This way it takes 16 bytes, compared to "string-stored" 36 bytes.

[@bashardlaleh2110]

IDK if my question is valid but in minute 9:00 it's not clear why you assume that reading a range of IDs from the visible index would be faster than the hidden index, why chances are those IDs being in one page is higher than chances of that being in the hidden index? doesn't this depend on how we are writing records? why writing in the visible index is next to each other but in the hidden is random?!

[@user-he4bl4tb4h]

Can you shard a clustered collection?

[@pemessh]

Quick question, why did they go with the recordid way in the first place?

[@hrmeet0509]

+1 on the same question

[@hnasr]

if I would make a guess, it’s technical debt. because of their original model when they first shipped MMAPv1. they had a single btree with a diskloc pointer directly to disk. that model is simple but had alot of problems mainly the use of mmap and didn’t have full acid support and MVCC . in 2014 they bought WiredTiger and that had the btree with the recordid. so it was easier to integrate is to replace the diskloc pointer with a recordid and keep all architecture the same.. otherwise it will require major rewrite it seems they did this big change in 5.3 as clustered collection

[@pemessh]

@@hnasr I see. That's interesting. Thank you for the answer.

[@burunkul]

why won't mongodb team make a clustered index a default one?

[@hnasr]

i envision it being default in few years once they iron out the bugs and limitations . which will makes it close to mysql innodb

[@oddym5788]

Where did you books and sword go :(

[@hnasr]

I moved office, they are on my side now 😄

[@JinKee]

Why is SQL so much faster than NoSQL?

[@stevefox7418]

Indexing, structured data etc.

[@Aditya24234]

That depends a lot on your workload, MongoDB can certainly outperform SQL by a huge magnitude provided that you have designed your schema that suits and fits NoSQL and similarly there will be certain workloads where SQL would run faster. A big chunk of that performance is also dependent on the configuration and the type of deployments you are running.

[@tonyhart2744]

You mean the other way around ???, most scalable database on planet use NoSQL, Vitess,Cassandra,ScyllaDB etc

[@jenkins9202]

In general it's the opposite, unless you're abusing NoSQL they should outperform any SQL database due to having relaxed ACID guarantees. You'll find most big tech companies had to eventually migrate to a NoSQL database because of SQL being a performance bottleneck when you're at a massive scale, e.g. Twitter, Facebook, Instagram etc. Of course it all depends on your domain, some use-cases require strong consistency guarantees with relational data which doesn't leave you with much choice but to use an RDBMS.

[@user-sv3bt7eh9d]

#bukopin #mandiri #britama #deposito greentea_metrimini@graharaya