MongoDB Schema Design Best Practices

Title: MongoDB Schema Design Best Practices
Date of stream: 4/9/2020
Slides: joekarlsson.dev/MongoDBSchema...
Have you ever wondered, “How do I model my schema for my application?” It’s one of the most common questions devs have pertaining to MongoDB. And the answer is, it depends. This is because document databases have a rich vocabulary that is capable of expressing data relationships in more nuanced ways than SQL. There are many things to consider when picking a schema. is your app read or write-heavy? What data is frequently accessed together? What are your performance considerations? How will your data set grow and scale?
In this talk, we will discuss the basics of data modeling using real-world examples. You will learn common methodologies and vocabulary you can use when designing your database schema on your application.
► $100 in FREE MongoDB Atlas Credits - Sign up here and apply code JOEK100 on the billing page.
joekarlsson.dev/free-atlas-cr...
► MongoDB Community
- MongoDB University: joekarlsson.dev/MongoDBUniver...
- MongoDB Developer Hub: joekarlsson.dev/MongoDBDeveloper
- MongoDB Community Forums: joekarlsson.dev/MongoDBDeveloper
► Additional resources:
- Modeling: university.mongodb.com/course...
- 6 Rules of Thumb for MongoDB Schema Design: Part 1: www.mongodb.com/blog/post/6-r...
- Data Model Design: docs.mongodb.com/manual/core/...
- Data Model Examples and Patterns: docs.mongodb.com/manual/appli...
- Building with Patterns: A Summary: www.mongodb.com/blog/post/bui...
► Subscribe:
Joe Karlsson's KZbin channel: / joekarlsson
► Video archive:
Source files: github.com/JoeKarlsson
Follow for notifications: / joe_karlsson
► Follow Me
- Twitch: / joe_karlsson
- Twitter: / joekarlsson1
- Website: www.joekarlsson.com
- Instagram: / joekarlsson

[@alejandromedina649]

I'm new at using MongoDB and this video has solved many doubts I had about how to design the ideal schema for my project. Thank you very much!

[@JoeKarlsson]

This is the best thing I have ever heard - thank you!

[@DIZZLEBOI44]

@@JoeKarlsson have a mongodb/admin job coming up

[@JoeKarlsson]

@@DIZZLEBOI44 Nice!

[@Ikilledthebanks]

What’s the easiest way to manage streams from mongo into snowflake using Kafka. The data structure changes and the update replace does not include data elements of the prior record. We require only the most recent update to the record no history

[@yanmoenaing71]

Rule No.1 is my favorite. => Favor embedding unless there is a compelling reason not to.

[@InternetLiJo]

Joe you’re a great instructor. Starting with the context/use case and then referencing the parts is so appreciated. Your channel is wildly underrated. Also love the humor references.

[@JoeKarlsson]

Thank you so much. I really appreciate that.

[@sarahahmed2112]

Thank you so much for posting this presentation. I appreciate the multiple recaps throughout the lecture, helped me a lot. And I appreciate the many resources.

[@BHVampireLF]

I didn´t wanted to watch a 1 hour video, I think is the first time I do something like this, and wow. I learned more in one hour than in 2 days readding weird written tutorials, guides and short videos. Thank you very much.

[@JoeKarlsson]

Oh my gosh! That's awesome! I'm so glad it was useful for you! :)

[@epacke]

Excellent presentation, it served super well as a foundation for the design discussions about our NoSQL schema. Although we went to with CouchDB all that you said felt relevant. Instant like!

[@axelneumann8443]

Summary: With MongoDB you have to consider your access patterns not only for indexing but also for schema design. There are three options for child documents: 1. Embedding 2. Array of references in the parent 3. References in the children (foreign key)

[@morapedikhutwane7405]

Thanks a squillion Joe! 👊🏽 This video is very informative, helpful and enjoyable 😊

[@JoeKarlsson]

This is my favorite comment :D

[@MohamedAsfer]

WoW. Really enjoyed this session. Thanks for the effort

[@JoeKarlsson]

Thank you so much! I am so glad you enjoyed it!

[@karamjeetsingh7637]

This was super helpful, Cleared many doubts I had. Thank you very much!

[@JoeKarlsson]

That's amazing! Thank you so much for watching! :D

[@abdulrashid2023]

One of the best video about schema design in mongoDB thanks @Joe Karlsson. I subscribed

[@khasmeenusman3931]

No one explains mongodb schema as you did. Thank you man. I finally understood MongoDB

[@winfle]

russia is a terrorist state, btw

[@jmgomw7787]

This is a very cool, well explained and fun talk on mongodb. 100% recommeded!

[@daminirijhwani5792]

this one video explained more than a couple videos combined I watched before I stumbled here. :) I have subscribed.

[@JoeKarlsson]

Oh my gosh! Thank you! I'm so glad it was useful! :D

[@compateur]

I'm sorry, but this sounded like a sales pitch :) Understandable since you work for MongoDB. MongoDB has a predecessor, namely XML databases. MongoDB has a lot in common with XML databases. I used to work a lot with XML databases and my experience is that document databases are a * bad * fit for business domain models. You talked about cons and you didn't mention the most important con: update anomalies. But wait a minute. You mentioned this problem and came up with a beautiful solution: a reference. And even better, there is something like unbounded references. I would say, keep evolving and you end up with an SQL database :). In my experience, document databases are great when you store documents as is. For example when you have a message warehouse, or for logging/auditing. XML databases are also used by publishers to store books (DocBook and DITA for example), newspapers etcetera. But using a document database for your business model. That's a bad idea to my opinion. I have the experience to use a very performant XML database as the enterprise database for a company and it had very nasty legal consequences. It was just not the right fit . A relational model is far better. It is also much better to have a schema for you business domain model. I would get very nervous if a developer would store its documents without a schema. No rules, really? Is that a positive thing for your vital business data? Good luck!

[@OfficialGOD]

I'm using mongol for logging and analytics

[@vinfern27]

When it says no rules. It means you define the rules that fit your business rather than let the rules define your business. you can enforce rules at the db level

[@nicolascondrea8068]

You deserve more subscribers! Flawless video, definitely learned a few gems from this one!

[@JoeKarlsson]

Thank you so much Nicolas - I may be biased, but I totally agree ;)

[@muhammadalirasheed8894]

Solved all my doubts man since morning was searching for embedding design. Thanks a lot 👍

[@JoeKarlsson]

That's amazing! Thank you for sharing!

[@rallysahil]

Thanks a bunch ! It was quite useful especially to see patterns at last.

[@andrew-ni1lb]

Dude, thanks, I have enjoyed your explanations!

[@bartoszmazur5168]

Two way referencing is just a PURE GOLD!!! Now it seems obvious, but I was trying to solve similiar problem without that 2WR knowladge 🙈, and made no progress for few hours, THANKS MAN!

[@pedropalma3748]

Great presentation, loved it. Keep them coming😊

[@HSBTechYT]

Discovered you from Twitter. Amazing talk Joe 🙏

[@harujei24]

Thank you so much! I finally understand how to design my Schema. I subscribed.

[@JoeKarlsson]

YAY! That's the best - thank you so much!

[@toneking972]

Super helpful. I was on the fence with PostgreSQL and MongoDB. But this convinced me to go Mongo

[@JoeKarlsson]

This is the best feedback - thank you so much!

[@anuragtiwari3032]

Thnx a lot man , this answered so many of my doubts. Deserves 10 million views, but we all know the world 😉

[@JoeKarlsson]

🙌

[@ardalanhadizadeh1799]

what i learned after just this single video was 10x times more than a 200pgs doc i read. thank you

[@JoeKarlsson]

I got you :)

[@joaquindelprado133]

this is a really nice video!! thanks a lot for sharing your knowledge it's help me a lot

[@JoeKarlsson]

Yay! Thank you so much! I'm so glad that it's useful :D

[@attilathehun1561]

Nice Video , Every thing is crystal clear you saved my lots of time , i am new to mongoDB from MySql background keep it up.

[@JoeKarlsson]

This is the best compliment! Thank you so much!

[@AngeloMiranda1999]

Great informative video to get started with NoSQL when you come from a relational DB background. Thanks!

[@JoeKarlsson]

I got you! 🥰

[@DaDa-gr7cy]

Great video, these intermediate videos are what youtube is lacking

[@JoeKarlsson]

Right? I feel the same way - lots of beginner level content, but not much for people beyond that. So glad you enjoyed this video!

[@victorkingma4371]

Excelent tutorial, gave me a entire panoram of mongodb

[@rajivraghu9857]

Excellent Joe!! loved you presentation. Really helped me to clarify some doubts on mongo .

[@JoeKarlsson]

I'm so glad to hear that!

[@xKenshin13]

Actually quite helpful. Answered most of my questions. Thank you :)

[@JoeKarlsson]

That's awesome! So glad it clicked with you!

[@splendorman7922]

better than official mongodb videos and tutorials.. thanks!

[@arthurgontijo1]

Thank you! Great content, it helped me a lot

[@extremespartan117]

Fantastic and straightforward presentation! Thank you sir!

[@JoeKarlsson]

Oh my gosh! Thank you so much!

[@prajunathunt]

The revisions at the end was helpful. Thank you

[@JoeKarlsson]

YAY

[@josemurillo4728]

Just one! Love that part. Great vid thanks!

[@JoeKarlsson]

I got you

[@t_kien]

I'm newbie and I found this video is amazing, now I can do it.

[@vaibhavrana4953]

I am new to No SQL and coming from RDBMS background. I enjoyed this session and it gave me very good idea about how to design Mondo DB.

[@gompro]

Thank you for bringing up a really cool video!

[@JoeKarlsson]

Yay! So glad you liked it!

[@soumabratabhattacharya3375]

Awesome tutorial. Thank you very much!

[@andreornelas]

Really enjoyed your video. Was kinda funny when you said MongoDB has no Rules but then we have a section with Rules. hehe. Altough they are very helpful to keep in mind. Thanks for the great content!

[@JoeKarlsson]

hahaha - lol - I def lied there! :P

[@fernandoventilari839]

You really saved me. Thanks a lot!!! Subscribed

[@louierichardson123]

I am creating a bug tracker web app and this has answered my questions about how I should structure my many to many relationship, so thank you!

[@JoeKarlsson]

That's so cool! Can you send me the code? I would love to check out what you're working on!

[@here2thrive]

1:20 -- Already I see you are a genius. Well done with this idea.

[@Cons2911]

I’m early into the video but I just want to say thank you. I’m new to this, so I’m trying to understand esp coming from relational

[@tgerambio]

This is beautifully explained. Thank you sir!

[@harrisfpv]

That was great content!!! Answered many questions.

[@JoeKarlsson]

So glad it helped! :D

[@joseluissanchezortiz3599]

Thanks friend. You explain very good. from colombian, Im speak spanish and I dont have seen any video so well explained in Spanish

[@gurmukhsingh2026]

This is the best into for MongoDB. thanks a lot!

[@JoeKarlsson]

I got you

[@spacemanjack777]

Thanks for a good video. Coming from many years with SQL I can clearly see some benefits with NoSQL. The main selling point for me is the loose data structure of the individual documents as well as query speed when everything you need is embedded in a single document. This is where NoSQL really shines over SQL Maybe I'm wrong, but I get the feeling that in most cases we still need traditional SQL-like relationship references between decoupled documents and quickly we end up with something that looks an awful lot like good old SQL.

[@PuzzlingQuestions]

This was really good. One thing that made me laugh is in the begining you stressed that there are no rules and then later you have "Rule 1: favour embedding" haha I just thought that was funny. Thanks for the presentation, it was helpful!

[@JoeKarlsson]

LOL - you're totally right. I totally missed the irony! hahaha! :D

[@edwardmike7523]

Thanks for the explanations, very clear .☺

[@sayitlikeitis1202]

Very well explained .. Joe. Super presentation .. Thanks much !!

[@JoeKarlsson]

You are so welcome - I am so glad it helped!

[@mahendranath2504]

this is a lot of clear content on how to schema our mongo, I have been checking about this topic for a long, but I have a question , in SQL we have partials if DB is big or sharding , do we have such thing here

[@redalaraf5211]

Thnx bro, real GOAT 👊🏼🤙🏼

[@biomedicaltechworld-btw8447]

Thanks for making such a comprehensive video for us. Stay Blessed.

[@JoeKarlsson]

I got you! 🥰

[@biomedicaltechworld-btw8447]

@@JoeKarlsson started watching MongoDB University course after this video. Your company must be proud of you. 👌🧡

[@andresbonilla475]

one of the best explanations, thanks from Bolivia :D

[@JoeKarlsson]

Oh my gosh! So glad it's useful!

[@YoloMonstaaa]

You're an amazing teacher. Thank you.

[@JoeKarlsson]

Thank you! 🥺🥰👑

[@quasarkid4256]

I’m really glad I watched this video thanks a lot

[@jrnoh]

Thanks man great video !!

[@dwhlab5436]

Such a amazing video .. many thanks .. great work.. cheers :)

[@JoeKarlsson]

Thank you!

[@johnlloydg.trinidad8267]

this is gold! thank you so much!

[@JoeKarlsson]

Oh my gosh! Thank you so much! So glad you enjoyed it!

[@haishuyao4834]

Really helps doing my course project and many thannnnnnks!

[@JoeKarlsson]

Oh my gosh! So glad it's useful!

[@Adam-uu8dc]

Love the office meme. Great tutorial!

[@davidtorres5012]

Awesome! Great explanation

[@premprakash7981]

Awesome Explanation. You have nailed it

[@JoeKarlsson]

Thank you so much! Glad you enjoyed it!

[@siddharthskumar6831]

Most underrated talks..awesome...👏👏👏👏

[@JoeKarlsson]

Oh my gosh! Thank you!!!

[@ranylfoumbi4641]

great! courage bro! it has solved many of my doubts

[@JoeKarlsson]

YES! That's awesome!

[@omadoyeabraham219]

Absolutely amazing explanation 👌

[@_dontlookup_4774]

Great video! I learned a lot, thank you!

[@JoeKarlsson]

I gotchu boo!

[@ashantharindu8915]

Mongodb scheme design is well explained. Thank you very much.

[@JoeKarlsson]

I got chu!

[@ukrolelo]

Thank you,very explanatory!

[@ayushkushwaha171]

Thanks for the video Joe

[@JoeKarlsson]

Oh my gosh! I am so glad you enjoyed it!

[@aniketsao]

Awesome video. thanks for creating and sharing it. I'm new to the NoSQL databases, is there a way to test the performance of embedding vs referencing ? given the unknowns of how an application would evolve, its hard to stick to one design. have you encountered any scenarios where over time the design needed to be changed from embedded to referencing.

[@JoeKarlsson]

The best way to determine which schema is "best" is to run experiments by running queries with different schemas and using the built-in MongoDB Performance Advisor to determine which one is fastest. And yes, I have seen people needing to shift schema's overtime all the time. It's really common for web apps to have features and requirements shift over time. So, an old schema that worked great, no longer fits new requirements. It's not a problem to update your schema.

[@umeshwar21]

great , I found it is very help full video to start project with schema design

[@JoeKarlsson]

I got chu ;)

[@SunilPatil-hs8wd]

Thanks a lot for awesome content. Subscribed to your channel

[@JoeKarlsson]

Oh my gosh! Thank you so much!

[@dgillies5420]

Love the 1960's-1970's Campagnolo Record bicycle de-exploding at around 25:20 into the video!

[@laurencedale3854]

This is awesome. Perfect education to meme ratio!

[@soyebganja8632]

Many thanks:)

[@saibabaalapati8446]

I'm relieved bro this is best and helpful very much thanks ....

[@JoeKarlsson]

🥰🥰🥰

[@hoquang6033]

super useful! Thank you man

[@lazareric]

I love you man, not many people talking about mongo, with proper knowledge of it. Good on you. An interesting question maybe, client wanted an architecture where each A collection item has many B collection items attached to it by reference, BUT querying A collection requires searching based on B collection items' properties, it was super slow, whenever I queried (50m+ documents), a collection based on properties of a different collection (so match, lookup, and the match on the lookuped documents), the query was super slow. The weird part is lookup was relatively fast even, but than the match on the lookedup documents' properties was weirdly very, very slow, any ideas why that happen? It's weird as I imagine once lookup is done, each document gets that embedded and then goes into the next stage (match) in this case, so why was match so slow, as it was supposed to match based on the already available properties embedded in the document in that stage with data in the stage (or maybe I don't understand, and it doesn't really work like that what I explained), it didn't have to use an index or do a collection scan, literally just filter documents in the stage, that's the only mind boggling thing in mongo that i don't get, everything else I love

[@ebinxavier8589]

Well explained.. easy to understand ✌️

[@ubermansch4871]

you deserve 1 million subscribers

[@scienceblossom6197]

I really enjoyed the it Joe! I just want to ask one thing, for one rule you said arrays should not grow without bound, and then in the example, it was just that! The two-way referencing in the task management app example where both users have an array of references of tasks, and tasks have an array of references to the users. So this has no bound... maybe by "without bound" you meant "extremely large"?! Thank you so much again!!!

[@anudeepananth]

Alright the information shared here is gold....

[@aminatawillane5633]

Hey there, best video ever on mongoDB schema design I ever seen that far...Thx for it... Since I am building a 5G SA tele medecine project based on mongoDB Atlas and Flutter, may I know please if u' re about to make a class for it? N thx again for that brilliant video...

[@knightx9405]

you are simply the BEST BEST BEST bru!!

[@JoeKarlsson]

I got chu

[@arunkutz]

Very Informative . Thank you for this video.

[@JoeKarlsson]

My pleasure!

[@juancruzstingo]

Joe, I have a small question, how should I handle updates in the twitter example if one of the followers changes his name? Thanks and the video was really helpful!

[@ajaypratapsingh2222]

Very well explained

[@code2287]

Nice resource.....could we get a complete mongodb schema playlist

[@ashishchandwani9179]

Awesome Video! Thanks, Can you also please make a video on Querying the Data from Documents, especially when we are using referencing.

[@JoeKarlsson]

That's a great idea! I can for sure do that!

[@makari1506]

Hi Mr.Joe, Great video and I learnt a lot. I was wondering whether you’ll be able to clear a doubt I have. In my project, users and tasks collections. I would like to save who created a task in a “createdBy” field. It’s always gonna be one value and would be selected every time one or multiple tasks are fetched. According to this video, I should embed essential data such as (userId and name) that my application would display. But I want to “refer” the user data so that fields such as name are fetched correctly. I worry if I embed the name and user updates their name, it could become costly to update in all the embeds. May I know what your approach would be? Thanks

[@tutym2]

loved the references

[@JoeKarlsson]

Glad you liked the memes!

[@arrafiialfa4609]

hi I got a question would it be better to keep reference on both document in one to many scenario or would it be better to just keep the reference on the one document that holds many docs ?

[@princematthew2145]

Fantastic Joe. Great Learning. When you talk about embedding example regarding product and parts, you favor parts. But you keep referencing to the size limit of the document. Yes its a problem, But more than that there is a bigger problem of consistency. If you embed a part and If you update the part from a screen, you have inconsistency. When you show the product you will show the old values. I am curious as to why you are not mentioning about inconsistency? I am a newbie in nosql. heavily influence by relational theory. Eager to know your thoughts

[@distributed-systems]

12:32 Embedding 17:10 Referencing