How to Structure Your Data | Get to know Cloud Firestore #5

[@9642arvind]

Hi Todd from 2020. You might want to add that we can now search for multiple attributes in an array using the 'array contains any' query! Great videos!

[@pagevpetty]

YES! This is exactly what I need! Thank you for sharing @Arvind Shekar. @Firebase Todd 2022, please update and with lists of pros & cons. Thanks :) if you even read these comments... I notice no replies.

[@AmeliorScout]

@@pagevpetty Page.. haven't I seen you in the Academind comments section? :D!

[@pagevpetty]

@@AmeliorScout Probably... from a while ago.

[@s3nsist4r]

Love this kind of tutorial. No code, just explaining concepts of how to structure data. Looking forward to the security rules video.

[@pagevpetty]

With visuals :) Visual examples (note squares) work!

[@davebeazer4007]

Love how you're keeping these videos current. Well done 2019 Todd

[@SigmundJaehn]

But where’s 2021 Todd? Is he ok and are there no updates?

[@lukegordon4734]

This is one of the best technical talks I've seen. Super clean, super simple to follow, and really explains the trade offs. Thanks for making Firebase best practices/structure easy to understand!

[@pagevpetty]

It would also be nice to see bullet lists of pros and cons for each.

[@dashandblast]

This series is the best. These last two videos in particular have been especially helpful, addressing questions I've been thinking about for months. I look forward to the next installment.

[@PeterFurstenberg]

Trying to make sense of the firebase universe, this has been a massively informative series so far in understanding how it all works! So Thank you. PLEASE update the series with understanding permissions soon!

[@eyezonmy6]

big fan of the teaching style, going over things more than once

[@joemac84]

Very informative video and thanks for keeping the content up-to-date. Love the discussion around best practices too because these things can be hard to find!

[@Firebase]

Our pleasure!

[@garypiong3752]

Please make more videos on this series; they're all so useful!

[@theanimalix5877]

Well done Todd 2018 and 2019, love your vids. Can't wait for Todd 2020! :)

[@almonddaves]

These videos are honestly such a treasure. They're fun, informative and easy to understand. There's a huge problem with the captions once Todd from 2019 appears. Not cool for people who need the captions. Still, 7.9/10, would recommend.

[@RayDaly]

Hope you do How to Structure Your Data Part 2. This one is great. I'd like to see coverage of using local storage IndexedDB etc with Firestore, Performance and displaying results, very large versus very small query results. Data Structure is such a critical part of app development.

[@jeffodle7810]

Do the queries at 11:05, querying documents where a property of a map exists, still work with Firebase 8+ ? I cannot get them to work.

[@moomoomamoo]

The example at 13:23, when we get the Favorite Restaurants, would it be a bad idea to use composite ids of the User and Restaurant documents for my FavoriteRestaurant documents? Assuming I have more Users than Restaurants, I would use something like ${user-id}_${restaurant-id} for my FavoriteRestaurant id. Assuming my Users and Restaurants use Firestore generated ids, my FavoriteRestaurant documents would be somewhat lexicographically close as there would be clumps of FavoriteRestaurant documents with a common beginning to their ids. How likely will this structure be at risk of hotspots? Also to give a use case for why these predeterminable ids would be useful is handling transactions over my FavoriteRestaurant documents for something like "store credit". What would be your advice on using composite ids and trying to handle this use case?

[@jesselima_dev]

Very nice video! Important concepts about how and when to apply which data modelling approach. I totally agree that we as developers need to know what data will be used in the application and how it's gonna be used. Also what kind of queries are inside the gather requirements for the app. Once you know what data you will need to retrieve and how often you will do it, then it will more clear about what model to use.

[@u2blr]

This tutorial is so good, even you are not using firebase still a good video about how to structuring data and designing system

[@remib.8524]

This is a great series of videos! Clear, fun and very informative. Thanks!

[@haraldlons]

Great video, love it! At 08:45, I like the solution, but how can you query "get all restaurants where userID_2852 is a key in the private_data.roles map?" I feel this is a very common user case, but don't quite get how to do it. Would really appreciate your help here!

[@OmerVexler]

Thank you Google! So elegant, Firebase is just awesome tbh.

[@rammalvarez2664]

My advice for Firestore data structure: Use both, remember if you have a hammer not everything becomes nails (this phrase it's better in spanish XD). But an interesting point to notice it's implementation and mantainance time, with top level collections tends to be easier and faster for me, specially if you reuse components in Angular. Edit: This becomed more useful when Firebase introduced 'array-contains' queries

[@cesmejia9014]

do you mean use both subcollections and top level collections?

[@martinmori9848]

Thanks for this tutorials, the real life examples are great and helps to understand everything in a simple way

[@sarensmith]

♥️♥️

[@DJNiems]

OK so one of the rules concerning arrays was that you cannot reference elements by index. So assuming that one chooses to store favourites using the structure at 12:10, what would you do if you say, wanted to remove (i.e. "un-star") an item at the second position in your list? You cannot simply delete the item at index [1], since that is not supported. Would you instead need to instruct the database to simply replace the list with the new one constructed on the client side? Or instead, do the array operations support edit-by-value, e.g. "delete item that matches 'rest_5153'", and by virtue of real-time synchronisation, the item removed in my UI is indeed the second item, since one would assume the ordering is synchronised as well?

[@burrfect_espresso]

Great video, so helpful in getting acquainted with structuring data in firestore. Question about your final example on using a top level collection of favorites. What are the upsides/drawbacks of tweaking your stucture by replacing the userid field on each top level favorite with an array of all the userids that favorites that restaurant (so each favorites restaurant appears in only 1 document with an array of users who have favorited it contained within?)

[@rabiahamouda8747]

Thanks for this amazing video and for the whole series! it helped me a lot, looking forward to the next video about security rules.

[@amrmonzir]

Yes yes agreed XD

[@tonyalbanese6446]

Thank you for this video. I am new to development and often doubt how I want to do things. Watching this reminds me that that there is often no one best way to do things.

[@zoulkifiroubarizi8213]

I realy love your turorials. Thanks too much.

[@Themusicaligera]

I have a question... The example at 13:23, when we get the Favorite Restaurants, we're getting all the document, including the user_ids that have favorited that specific restaurant, what happens if a LOT of users have that restaurant as a favorite one, doesn't it exceed the indexing limit? and also isn't it useless to get all the users that have favotired that restaurant, when that field is only to query information?

[@WolfRapMotivacional]

Firebase, React, Material UI and Typescript is awesome!

[@aaronhong4345]

Love the crash course vibes

[@glrbrasil]

This is so insightful Todd, thank you 🙏

[@nayankurude4129]

I am a miser when it comes to like a video.. But you are gonna get my like.. Keep the good work up.

[@mayankkataria7080]

Should my single document contain only one chat message or an array of all chat messages ?

[@trod-123]

I like the top level collections solution brought up at the end. That said, if I am understanding this correctly, by introducing denormalized data and using cloud functions to update all copies of that data when the original data changes, this means more data write operations that we'll be charged - one write per copy. Right?

[@jasontissera4331]

Love that there's no limit to the number of documents you can have.

[@BauldyBoys]

Always glad to see Todd from 2019.

[@akikoobodo]

best teacher ever!!!

[@i-heart-google7132]

12:28 - am I missing something? How would we create the FavoriteRestaurants top-level collection? One restaurant can by 'favorited' by multiple users, so how do you set the "user-id" in each document of this array?

[@julianalvarez3878]

Hi @@ToddKerpelmanCorp! First of all, excellent videos series, very helpful! I have one doubt regarding the favorites use-case and the top-level collection approach. Is there any way to avoid someone creating duplicated documents? I can only think of creating a composite key for each document, something like restId_userId, or having a cloud function monitoring each creation in order to eliminate any duplicates. Are any of these the recommended way to accomplish this? Thanks

[@gmacv3777]

How do we store the array(favorites) of objects shown at 10:30 in Android?

[@adamryason5509]

One thing I'm still trying to figure out is best practice for displaying the collection of "restaurants" that a user can browse. Let's say there are 20 restaurants, first option would be to query this collection every time the user browses (20 queries per browse) while a second option would be to query a document containing the list of restaurants and then just query the restaurants that they select (1 query for the list, then 1-5 queries for the restaurants). I would love to hear other's opinions!

[@ivanpambudi6913]

umm current day Todd, you're good to keep this up-to-date but you forget to update the subtitle too, not a big deal but sometimes it help me to better understanding

[@vincentdufour613]

So when you have a N-N relationship like the link "favorite" between Restaurants-Users, you create a new collection that contains the id of the restaurant, the id of the user and the favorite data. You are just normalizing the unnomalized data like one do in SQL with tables instead of collection ?

[@anoopisaac]

Hey Todd, thanks for the awesome tutorial. In my application I have list of users and all the pages that they own. There are minimal page queries that span multiple users, so following your recommendation, can I go with storing pages as a sub collection to users? just wondering whether the fact this its being 'User' object would have any impact on the structure?

[@greganderson8416]

What if you are doing the 'inverse' of a favorite, such as a 'done' flag to filter out completed items in a task list? Why are there no inverted logic queries in firestore to support something like this? Ex: an array-not-contains query. Even based on the earlier video where it was explained that queries use a binary search on indices to produce a contiguous sequence of results I don't understand. The inverse of that is simply the concatenation of the two contiguous sequences before and after.

[@hakanbuyuktopcu3720]

I love the videos, just want to point out that the subtitles are not updated for Todd from 2019. Thanks!

[@carsonskjerdal473]

This was incredibly helpful! Much appreciated, will be watching again.

[@TheL96]

Thanks, Great video. bring on the much needed security rules video pls!

[@YousefKabbara]

#askfirebase Thank you for this great series, I have a model like the following: Collection: Place String name String description Int numberOfVisits and so on Q1: for the first two parameters ‘ *name and description* ’ I need to make it *multilingual* what is the best structure to make it so while keeping minimal data downloaded and the minimal number of queries? Q2: you're talking about having a sub-collection for reviews, but what about having a third collection for the relation between users-resturants-reviews ? Best, Yusuf

[@johann_reimann_2077]

Can we say, in this example making multiple top level storage will be more efficient than storing data as the snippets or sub-level?

[@ptrchovan]

Denormalisation is against every rule I have learned in most db specially sql. I understand this is nosql so I have no other option then get used to it which I guess will be the hardest part at structuring db in firestore. What a mess.. :o

[@thelonearchitect]

It's a trade of read-performance against maintainability. Firestore acts more like a fast cache than a real database imo

[@gachugu5589]

Last winning structure. What if another user adds the same restaurant as a favorite? Does it not mean that the user_id field will be overwritten?

[@luiswebdev8292]

If we have a Users and a FavoritedRestaurants collection, where FavoritedRestaurants has user_id and restaurant_id, don't we end up with TONS of duplicate data? Imagine 1000 users favorite the same restaurant, then we have the same restaurant snippet in 1000 different documents. This seems like an undesirable amount data duplication. Is this a right approach to do things?

[@luiswebdev8292]

Okay after watching more videos I see that duplicate data is a thing in NoSQL and it's not so terrible as in SQL

[@alexiszorba25]

What if my App/Web is multi language? How do I store a list of Types of restaurants? Should I use Remote Config and store only the Ids in Firestore? Or should I create a different structure in Firestore for that?

[@dapkman9354]

Great video. thank you! So much useful information and it's so interesting to watch at the same time!

[@karolrvn]

Great, broad and deep; with many considerations taken into account.

[@SandeepKumar-xu7wd]

at last, if we create a separate user collection and favoriteRestaurants collection. If I change the restaurant address, I have to change all documents that match the same restaurant's id in favoriteRestaurants collection. And I think it's not a good way.

[@kagame6524]

Really great tutorial!

[@Firebase]

Thank you! Cheers!

[@ramirauch4515]

all these examples are easily resolved with relational db. remind me again, why use nosql? also in the proposed solution you present separate options of use, but what if you need this and also that?

[@rahistaffasia6261]

This video made me felt NoSQL has so many issues. The more I watched the more awkward I felt. This felt to me like a video on how bad NoSQL is

[@fatemebahrami7309]

love this tutorial! besides that, just wanted to let you know that the captions are not synced due to Todd 2019 intteruptions 😆

[@Themusicaligera]

I have a question on how to update the snippet reviews on 5:29

[@thedevguild7525]

Hi the transcript and captions seems to be off! it seems to be lagging behind

[@prabhunerahul]

In firestore, can we add new json object in the same document instead of creating a new document in the collection every time user adds as review? If I create separate document for each .add json object, I would have 1000 document reads per day per user which would be expensive. What would be the best optimum solution?

[@AnkitMaheshwariIn]

Single query required many times to fetch collections and sub-collections together. Is there any plan in near future?

[@goleary]

Am I the only one who has mis-aligned captions & subscribe / watch more content at the end of the video. I'm guessing it's due to the "Todd from 2019" segment being added after the fact. It's like this in many of the videos in this series.

[@cyber3nai1]

Problem with subtitles from 2:37 :(

[@ahmedelsobkey2535]

are you find a solution for subtitles problem?

[@ahmedelsobkey2535]

I solved the problem with selecting second choice in CC (English auto-generated)

[@1002CK]

Tod, can u explain about using cloud function in Cloud Firestore ? #AskFirebase

[@simonleichtle6913]

Now I am curious for the cloud function video :D

[@zechcampbell]

Where can I find the video about Cloud functions you mention? Interested to see how to use Cloud functions to keep duplicate data up-to-date across multiple documents.

[@BrentBroome]

I think that video only exists in the future (for now). Sounds like one of these on security is next, then maybe functions after that? Not sure of the team's roadmap for this series, but Todd has alluded to functions being the topic of an upcoming video a couple of times. (Someone please correct me if the functions video is already out and I've just missed it.)

[@nieldv1]

Hi, I must say I love your videos. I'm using a Firestore database and building an app that contains users and clubs, and a user can belong to multiple clubs (ie a club can have multiple users) and a user can be an admin of a club. So a club user that is an admin should be able to see and manage the users of a club, and a user can join clubs and see the details of the clubs etc. I come from a relational database background so naturally I created three top-level collections - clubs (containing club info), users (containing user info), and users_clubs, just containing the combination of clubid and userid and an isadmin indicator (eventually i will proabably want to add more roles/permissions on a user_club level). I'm wondering if in your opinion this is the best option or should I add a subcollection of clubs under users to list the clubs they belong to, or should I add clubs as a map to users (with clubid and role information)? I realize there's no one correct answer to this question but I am trying to minimize cost in terms of storage and queries. Thanks!

[@kalmahiks]

video and subtitles don't match after insert fragment from 2019(

[@Phoony17]

Excelent video, right now I'm building an web app an this is extremely helpful to get your thoughts in the right direction

[@runfunmc64]

Very helpful, thanks so much!

[@varunrao2135]

This is very very well done

[@lucasluizss]

Great video!

[@noamgonen6243]

14:07 - talking to your imaginary friends eh? I thought I was the only one....

[@oghenefokethompson7407]

Love this kind of tutorial! greate job!

[@divertechnology]

denormalized is the best. memory save is from times where memory was hard to have

[@3litepker]

Can anyone tell me why you just wouldn't store *only* the restaurantId in the favourites array of the user collection? I'm pretty sure you could just query all the restaurantId's in the users favourites and pull all the data that way.

[@a1r592]

Coming from SQL this feels kind of hacky but i guess it's a different way of thinking. Great video nonetheless; you explained it really well.

[@DavidChoi-wz5pn]

Can someone explain to me how 2019 Todd appears in a video published in 2018?

[@josefsantamaria]

Its really helpful with this kind of tutorials, why not do an updated version in firestore about event and members etc that you did for realtime database!

[@afolabiaiyeloja5315]

Great video! I'm considering using GraphQL with Apollo on the client side and was wondering if it's a good method for fetching data. My stack is React | Apollo Client | Apollo Server | Firestore. Thanks for the help :)

[@nevinjohn]

Hello Todd, I am developing a flutter app and I have this "news" collection with documents pointing to each article. These documents have the field "viewed" which is basically an array that contains the uids of all users who have viewed the news article. I want to list the unread news to the user so I was thinking of setting up a streambuilder to query for something like .where("viewed", "!arrayContains", "uid") However, this is not going to work for obvious reasons. Could you please shed some light on the possibilities of a data model so that I can look into setting up a streambuilder that can populate a listview of unread news for the user? I have the top level "users" collection with essential user details.

[@rajeshsharma8084]

do queries support ! operator

[@ExequielGomez10]

Someone knows wich fuction of firestore is for keep the spnipet in sync?

[@Frank_van_Puffelen]

To keep a document or a set of documents in sync you'll want to use realtime listeners, as shown here: firebase.google.com/docs/firestore/query-data/listen

[@jedcaluag2319]

Thank you!!! Very helpful.

[@camillo8575]

Really good tutorial. Isn't putting everything in top level collections like using a SQL-like database structure?

[@nataoliveira2247]

kinda, except you duplicate data very often

[@Rave1502]

Very nice video. Just adding one more use case to the given examples, please help me understand what would be the preferred approach. Now Along with user's favorite restaurants, I miraculously also receive user spending per restaurant. I want a report of individual's average spending per restaurant. And also a restaurant's per user per-month/year/etc.

[@shamilmammadov8246]

Just perfect! Big thumb up!

[@stevengao8345]

what is differences between firebase original database and firestore ? which is better ? using firebase now do I need to learn firestore ? I mean once firestore comes out of beta , will original firebase got deleted / removed ? thanks

[@karimjordan394]

steven Gao There already is a video on that. Just look it up.

[@markusgattol]

1) firestore - the new NoSQL store 2) realtime db - firestore's predecessor, no NoSQL store but rather JSON store (quite good for storing text in all kinds of formats though) 3) firebase storage - used to store non-text e.g. blobs such as images, audio files etc. overall, if you have a normal project that needs to store stuff, you'd go with firestore and firebase storage these days

[@levanimelikishvili7691]

Firestore is new and preferred db if you use firebase. The old realtime database is basically one big json object while in firestore you have collections and subcollections to better organize your data. You can read more about how these two compare here: firebase.google.com/docs/database/rtdb-vs-firestore. P.S. I don't think realtime database will ever be removed as it's used in many applications already.

[@mika2666]

No, the realtime database is a different product and won't be discontinued if firestore releases for the simple reason that people rely on it

[@iceniveth]

Ya might wanna read this: firebase.google.com/docs/database/rtdb-vs-firestore

[@bakanndy]

"You can't run 2 array contains queries on the same array", I don't really understand why you can't except that arrays are weird. BTW thanks for the video, this is the best one on NoSQL, you're a genius.

[@heyyy4987]

the winner 4 will charge (billed) me twice?

[@RobinYoulton]

Thanks Todd, this advice is right on the point for me 👍

[@rachelnguyen9966]

Guys, @Firebase How to use reference field here? For example, I have Posts and Users collections. in Posts, reference field is Author, link to user: users/userid When get the Post details, how to get the user details from reference without using another query like: doc(db, "users", userid). I do not want to consume 2 queries for that. Suppose reference field can return the user data in get Post query too.

[@rahulpanwar5344]

How to trigger firebase function inside from react via onclick button

[@AkhilKumar-ys3ph]

please upload all series

[@leeboyin945]

Anyone using Firestore must watch this video before start planning data structure!

[@lukmanahmed6999]

implementing likes or rating for favorites restaurant will be really expensive if it has been favorited by a lot of people. Assuming 40 million people favorite a restaurant it means you will have to perform a write for each person who favorited a restaurant. Is there a cheaper alternative.

[@codruterdei]

did you find any answer? I am looking to do this myself.

[@markusgattol]

why not use custom claims (stored on the user ID token) for role based access management instead of an array of UIDs? firebase.google.com/docs/auth/admin/custom-claims

[@MesinKasir]

Subtitles is not synched after Todd from 2019 shows up

[@adwinang4188]

The english CC is no longer correct after the correction from 2019

[@asdfghyter]

The last example: Here is a problem (many-to-many-relations) that NoSQL databases are generally bad at ...[long list of possible solutions] ... the best solution is to use the traditional SQL solution of storing the many-to-many relation in a separate table.