Maps, Arrays and Subcollections, Oh My! | Get to know Cloud Firestore #4

Is your Cloud Firestore data structure set up correctly? Cloud Firestore supports Subcollections, Maps, and Arrays, all of which can be useful, but it's sometimes difficult to know when to use which one. Why create a subcollection when you could just create an array of maps? Or a map of maps? When is it appropriate to use one over the other? And what would work best in your app?
In this episode of Get to Know Cloud Firestore, Todd does his best to get your questions answered and gives you all the pros and cons, so you can structure your data the right way.
Cloud Firestore Data model → bit.ly/2PZtQnD
More episodes on Cloud Firestore → goo.gle/Cloud-...
Subscribe to Firebase → goo.gle/Firebase

[@IsaacAsante17]

You guys have mastered the art of teaching.

[@codinginflow]

The animations are amazing

[@MrCamster999]

So true

[@eduardosalles9212]

NOOOO.. we learn by seeing the codes. Show us examples.. we want the codes..

[@AymanShaabanELsayedDEVELOPER]

Hey todd from 2019 you forgot to make a new caption for the video after edit

[@adamm3122]

Ayman Shaaban ELsayed DEVELOPER this was really impressive how they did this!

[@tylim88]

you can also switch to auto generate caption

[@klutch4198]

"This tends to put your app in a state I like to call 'uninstalled'" hahahaha

[@sheelajain370]

getting uninstalled

[@asdfghyter]

So the basic rule of thumb might be "Shape your data like your queries". Data that are usually queried together should go together and hierarchies that exist in your app should probably also be mirrored in the data.

[@GustavoDiSalvo]

Great video, thank you! I'm just learning and designing my NoSQL model and every one of these videos teach me exactly what I wanted to know. I have one complain: when you update the original video please update the subtitles too!

[@reycoseguma2184]

This is a very good explanation for Firestore. Now I know which I should use or not use on Cloud Functions by knowing these rules on Firestore. I think I can learn more by watching the whole series.

[@roquefore]

Love love love love love all these Firebase videos! Awesome! Keep going!

[@codinginflow]

5:50 Aren't 2 million reads = 1.20$?

[@anonwithamnesia]

yes and 2 billion reads are 1200$

[@tilakmadichettitheappdeveloper]

fawk !

[@rafidhoda]

Amazing video! That ending was HILARIOUS

[@sexyeur]

(2018, 2019, 2020...) Todd is cool in any year. Thank you for this video!

[@Klazyo]

1 - 1 mb per document 2 - you can not retrieve data partially 3 - Quering are shallow 4 - better put ur data top lvl if u want search thru all document or retrieve

[@boon_keng]

Could you please next time cover topic of timestamps, dates in Firestore / Cloud functions! I mean getting, storing and sending to clients! Thanks in advance!

[@michaelscothon9871]

,so

[@DanielSadjadian]

This guy is just brilliant and the ending was so funny.

[@toastrecon]

Is 8:54 a typo? I'm guessing that both subcollections should be named "characters". Thanks, this video helps!

[@KrishAgrawal]

So Todd can time travel. great work google 👍

[@marcelogamer3214]

Geopoint will have their own querys soon? Or not at all? Like distance between geopoints or querys by proximity? Really waiting for those!

[@michaeloosthuizen2383]

I would be interested in this too as I have a project that might need to calculate the distance between points. I am dreading having to look up that crazy formula and trying to figure out how make it work. The Geopoint object has a compareTo method that might be used for this, but the docs don't really elaborate on what it returns: firebase.google.com/docs/reference/android/com/google/firebase/firestore/GeoPoint#compareTo(com.google.firebase.firestore.GeoPoint)

[@MichaelEvanspishposh]

I’m running a side Postgres dB with one table that just keeps track of the Firestore Id and a point lat long. So I query the Postgres when I need geospatial but other 99% is direct to firestore

[@lukecross5439]

July of 2020, this is still not supported officially, but, i got filter my documents adding they on Adapter and using Collections.sort(); But, the disavantage is the realtime data not appears more.

[@samzmann]

Thanks for array-contains. This is really good!

[@frankhampusweslien]

So the Arrays are actually Sets? Why not call them Sets...

[@mika2666]

🤣 the shark tank refrence at the end

[@LuisSierra42]

ikr, that cracked me up

[@michaeloosthuizen2383]

I find that using sub-collections are great for data privacy because you can have different rules for a document and its sub-collections. Good data structures plus strong rules equals wicked privacy kung fu.

[@ecofriend93]

Security rules need to be taken into account when setting up data structures as well. I just found this out the hard way. Luckily I'm still in the dev phase so can pivot to a different data structure that allows me to set different security rules for sub-collections vs the parent collection.

[@Zakariat89]

This is one of the best explanations I've ever seen on data structures. Not just for firebase, but noSQL in general. Thank you.

[@qm3ster]

So, arrays should be used like sets?

[@-no-handle]

I think they just index the arrays as objects. I just tried adding duplicate strings to the array and it worked just fine.

[@bahadirbalban8773]

love the pace of these videos. All teaching videos must be made like this.

[@johnalto6517]

this guy is awesome

[@alexgu3257]

Caught a small error at 11:24, missing a "s" after "keyword". Thanks for making this btw.

[@GiammaCarioca]

Nice video but it would be great to have an article to accompany it

[@jasonsebring3983]

I keep thinking just use firestore for realtime updates but use another database for real search capability and return ids for firebase to use. Use the right tool for the job but requires keeping data in sync.

[@philip2.042]

does this mean that if a document only contains one array, the size of the array is also limited to 20,000 elements (like maps) as well?

[@acche-rc]

I'm surprised the video didn't cover one big limitation of ArrayContain query: they cannot be chained.

[@SingularityLabsAI]

What's the best way to store say I am expecting to store 10m keys. The workflow goes like this If I encounter a key in my app, I check firestore if that key exists, if it does I do some thing if not I put the key in the firestore. I don't really care for the value of the key if there is provision to delete the key or else let the value be true/false, such that I can use false if I wish to delete the key from firestore. I need any of key insertion/lookup/delete in O(1) or O(logN) .

[@bala-st9cj]

So expensive cloud firestore 😩

[@blessing7695]

I was asking myself this question just yesterday bcz we need to scale up our app.

[@AndreaLombardo84]

So, in the end, what's the best data structure for an Agenda like application to store customers appointments 😄😄😄? The app have a collection of customers and I need to show the full month appointments in a "Google calendar style", but also need to count how many times a client has an appointment with me. Any suggestion? Thanks a lot.

[@Fanaro]

Very helpful video. Thanks!

[@Firebase]

We really appreciate the positive feedback, Philippe! Thanks for watching 😀 If you liked this, be sure to check out more episodes on Cloud Firestore here: goo.gle/Cloud-Firestore

[@TedHopp]

The distinguishing feature of arrays is index-based access, which Firebase doesn't support. Seems like a better name for "arrays" would have been "sets" (or perhaps "multisets" or "bags" if repeated values are supported). Or am I missing something?

[@michaeloosthuizen2383]

In Firestore rules the arrays are called lists and object are called maps. I think lists would be a more accurate description of them, I am not sure why the difference in terminology between the data structure and the rules... maybe because arrays and objects are more web dev terminology and lists and maps are more "traditional" terminology (at least from my perspective) and probably more correct from a CS point of view.

[@TedHopp]

Interesting. Like arrays, the abstract data type "list" corresponds to a number of ordered values. Lists usually support, at a minimum, ordered traversal of elements (i.e., a "first" and "next" operation) as well as ordered insertion (e.g., "append", "prepend"). From what I gathered from the video and my (admittedly thin) knowledge of the API, Firebase doesn't support any order-based operations on what they call arrays (in this video) or lists (in the rules). Therefore, I don't think "list" is any more appropriate than "array" for what Firebase does offer. If they were to ask me, I might have suggested they use "map" and "collection", but they already assigned a different meaning to "collection", so that's out. Oh well.

[@michaeloosthuizen2383]

Iteration is supported, its just not supported on the database (as far as I am aware). If you use the Javascript SDK an array from the database is an array object, and a Javascript array is a list. Also the support methods of a data structure do not make it a list or not, the actual structure determines that. Another also: when you query a document you get the ENTIRE document, so there is no real need to iterate over an array or manipulate it on the database, you would do that in whatever code made the query (be it client side with a client SDK or API or server side with the admin SDK or API). For example, if you wanted to do server side manipulation you would do that with Cloud Functions using the admin SDK, not in Firestore itself.

[@TedHopp]

Thanks for the info, Todd. What you're describing sounds to me like a set implemented as an array which is in turn simulated using a map. But the exposed functionality sounds like a set, not an array or a map. Yes, values are entered by supplying them in some order (that looks like an array) and they are returned in some order (that looks like an array). However, from what I've read, duplicate values are not supported (the array ["a", "b", "c", "a", "b", "c"] gets stored as ["a", "b", "c"]). In this case (and in others) there's no order guarantee. From the api user's perspective, it doesn't have the characteristics one would expect of an array. I'm not sure I understand your comment. On the one hand you say I have indexed-based access and ordering, but on the other hand I don't have that because the only operators I have don't allow for indexed-based access or ordering. (For instance, I can't insert an item at position 3 if the same value already exists at position 0. Actually, I can't insert an item at position 3 at all, since the api doesn't allow it.) I understand from the blogs why the api is designed this way, and I think it's a good design. A true array model would have lots of issues. I just object to it being called an array when the only thing it has in common with an array is that sets of values are transmitted in both directions in an apparently ordered form (which is not guaranteed to be preserved).

[@nickadams2361]

I swear this guy is the only part of any google software I like.

[@meerimkubatova225]

How to get number value of nested object, I tried this --> Task task=FirebaseFirestore.getInstance().collection("Users").document(userId).collection("Award").document("Kilograms").get(); DocumentSnapshot documentSnapshot=task.getResult(); myValue= documentSnapshot.getLong("Kilograms").doubleValue(); It did not work, please help me

[@Quetzalcoatl0]

I don't know what is the best way of creating a voting system for example for "movies". I want people to vote from 1 to 5 stars but also keep track if they voted or not. I'm showing the vote count number and score of each "movie" so i want it to be in the "movies" collection. I can make an array of users in the "movies" collection, and add the UID there so i can check if the current user voted or not, maybe this is the correct way ? I don't think more than 1000 people are gonna use the web app i'm making, it's for a very specific non popular thing, so i think it should be fine even if there are 20k people in the next 5 years.?

[@БабенкоДмитрий-э3х]

Problem with subtitles from 6:23 :(

[@tilakmadichettitheappdeveloper]

use google voice translate

[@KavinduWijesuriya]

You're great but to keep up I have to slow down the playback speed :D

[@l.e.nichols9382]

me too lol... he speaks to fast.

[@RobinYoulton]

Thanks Todd, I REALLY appreciate these insights on Cloud Firestore.

[@nevinjohn]

I want to list the unread "news" if the "viewed" array within the news documents doesn't contain the user id... Any ideas on how to implement the logic "notarrayContains" ? I guess firestore does not support it? Thanks for any help!

[@nevinjohn]

For those who are wondering how to achieve this: groups.google.com/forum/?#!msg/google-cloud-firestore-discuss/wXn4nd2RBIc/jsWhxy3FBgAJ

[@kushalbaldev8490]

Anyone here who are struggling to get firebase working in angular 10? getting this error This likely means that the library (@angular/fire/firestore) which declares AngularFirestore has not been processed correctly by ngcc, or is not compatible with Angular Ivy. Check if a newer version of the library is available, and update if so. Also consider checking with the library's authors to see if the library is expected to be compatible with Ivy.

[@courageous-schools]

Just keep them videos coming!!!

[@klaudiaadamus3353]

hey, guys, just typing this to say your "subscribe" and "watch more" linked windows are popping up way too early - that may be because of your new content addition ;) anyway, a great video collection, thanks!

[@codewithlenny]

The video has come to an end and for that reason, I'm out!

[@pkop4]

If array_contains is like map field === true, why can we only query on 1 value at a time, and not multiple ?

[@digimbyte]

can this 20k document field limit be bypassed if we can turn off indexing?

[@l.e.nichols9382]

You speak so fast i had to watch this 3x before i understood the firestore hierarchy fully lol. Good stuff though, thank you.

[@CurtBathras]

LOL. I watched it at 3/4 speed which made him sound a bit drunk, but definitely easier to follow. Great content!

[@steakdriven]

Your product gives me a damn headache. There is no way to get around the fact that I need to make multiple damn queries if any of my fields are more complicated that a number or a string.

[@almeale317]

It could be nice if we could check if a date or number is what we expected(in rage or higher or lower) into a object in an array

[@Ballistah]

Great explanation! I LOVE that you also update the videos! That's great!

[@NathanNissen-i2v]

Didn't add array specific functions because it might throw an error 💀

[@marcelomms4021]

When to use simple string to store a reference vs a real reference (type) ?

[@Algardraug]

So... is the Charles Dickens app market booming yet?

[@DodaGarcia]

"In a state I like to call 'getting uninstalled'" 🤣🤣🤣

[@laxy1070]

hey but how can I get an array from the collection I did not find any documentation in a firebase site??

[@youmeandeveryone5893]

Ok, wheres the video about collectionGroup?

[@v_iancu]

8:44 Can you please point to an example of how such an index should look like?

[@vanyvanov]

Thanks, it really helped!

[@Firebase]

You're welcome!

[@benbenzyhome]

Do you have videos for firebase 9

[@weilieschok2001]

Could you make some tutorial if i need to store data historically? e.g. employee | date | department | job_title E001 | 1-Jan-2018 | D01 | Trainee E001 | 1-Mar-2019 | D02 | Manager department | date | dept_name D01 | 1-Jan-2000 | Sales D01 | 1-Jan-2017 | Sales & Investment D02 | 1-Jan-2000 | Legal D02 | 1-Jan-2017 | Audit * D01 was first introduced as Sales but later renamed to "Sales & Investment" in 2017, same applied to D02 From the relationship above, i know the employee first hired , 1 Jan 2018 and joined in 'Sales & Investment' department and not 'Sales' Then promoted as manager in 1 Mar 2019 and joined "Audit" department and no longer 'legal'. It's all very easy in SQL/RDBMS. But it's quite tough to model it in NoSQL. Hopefully there is some tutorial on these topic. Thanks

[@thedevguild7525]

Dear Firebase Team, I have a form which collect the -cafe name -latitude -longitude of its location. I would like to save the entry from my form into firebase firestore (using javascript) but have difficulty saving the lat and long as a geopoint (var name is coordinates). Need your assistance on how to code. form.addEventListener('submit',(e)=>{ e.preventDefault(); db.collection('cafe').add ({ name: form.name.value, coordinates: {Latitude:form.latitude.value, Longitude:form.longitude.value} //unable to save lat and long as a geopoint, need advice. }); });

[@GiridharPSPKPSPK]

I have a doubt regarding deleting an array of data in Arrays. The Structure looks like image: [ 0 : imageurl : ".........", isliked : "true", 1: imageurl : ".........", isliked : "false", ] now I'd like to delete index 1 how can I delete using the Cloud Firestore ArrayRemove method or is there anything else to remove based on Index position. Thanks & Regards :)

[@michaelblum4557]

Great video, thanks

[@Firebase]

Glad you liked it!

[@aspensareprettyneat4002]

"Wrap it up Charles Dickens, you are aren't getting paid by the word." -Gene, Bob's Burgers Also, still a great tutorial 3 years later.

[@devmeta1478]

What if I have situation where ,logically speaking, I should have one document with few subcolletions and each subcollection have documents with many subcolletions. And i always wanna show whole document or just root (top level without any subcolletions) But i know that many users are gonna write and change that document very often, so should I split it to top level collections to avoid write failing ?

[@louisart92]

Great!! and how can I do a query like "get all characters from each book into 'dickens_book'" I mean get all subcollections 'characters' from all dickens's books in orther to save them into a array after that?

[@n00bstude]

Thanks for the quick education!

[@erikpolygon5471]

all i understood was "oliver" and "dickens"

[@steverobertson2732]

Great video. I recently ran into the current limitation of not being able to delete collections (or sub-collections) so deleting a document with sub-collections would orphan the sub-collections. Seems like another reason to use arrays within documents instead of sub-collections - at least for now. Your thoughts?

[@orcun481]

@Firebase Where is the documentation of the new array features? I couldn't find in here -> firebase.google.com/docs/firestore/manage-data/delete-data

[@amanmanojpreeti]

So, i guess the laptop there was just for show! 😅

[@adityapalavajjhala1841]

Which among these is more efficient? Updating an array field inside a document or listener on a sub-collection with documents?

[@rijatru]

If I query a collection with 40 documents, will I get charged for each of those documents reads?

[@stofstik]

Very helpful vid! But only one write per document per second... WAT

[@PeriPC]

Great teaching experience guys, great job Todd. The only thing that i am skeptical all this time with Firestore is the limitations of upgrading. Let me explain better. We build an app today with specific needs, let's say for example a Charles Dickens' app where we need to get the books titles only. After users feedback we decide to upgrade our app and add "search by character's occupation" function. In these changes (upgrades) of our app, we need to re-design our entire db "schema" and maybe migrate old db structure to a new one... It sounds tricky

[@karthikshekar2715]

hey, how would I query if I had a collection called videos tat contain the video url etc and another collection called users tat contains the username which needs to be displayed during the video play.

[@praveenpushpkar57]

If a document can have just 20000 fields max, then its a huge problem to something like comments and replies. A document for comments can have fields for each comment with subfields for comment meta data(eg. time,comment text , comment by,etc). Further for each comment field(which is in the form of a map), can have another map for replies. This would easily and in no time consume those 20000 fields. If we use sub collections for each document or even each reply then it would just cost huge for reads as the number of comments and replies grow. Please tell me if i am missing something and the solution for this problem as till now firestore was my favorite, but now i feel i need to switch to something else :(

[@Frank_van_Puffelen]

Also posted as a question on stackoverflow.com/q/62453703, where Doug already responded. Unless you have a different question here, I'd recommend following up on Stack Overflow if you have further questions.

[@LarsRyeJeppesen]

Hey Todd from 2019, apps are so 2019

[@zizaart1199]

Clarified a lot of things, tysm

[@alfredgaillard2715]

collection("collectionName").where() doesn't exist in angular...

[@OMARVOLCOM1]

I can't find any documentation to allow me to update my array of maps in Firestore from flutter. Any ideas on how to find this?

[@alexanderhemming6148]

Does anyone know why pressing enter in a material UI text field submitting info to firestore does not add enters/ returns into the database?

[@Shikschauhan]

Is it possible to update a map in an array ?

[@cemmer1295]

getting uninstall was hilarious

[@Cindorqw]

arrayUnion() still doesnt work properly

[@jayantnarula1157]

If any one tell me , How to use 'Forward Slash ' as a character in document Id in swift

[@qm3ster]

How can I prevent indexing? Should I store a serialized JSON object (as firestore String)?

[@TikTok-iv5fe]

hey guys,is there any one have information on flutter on this firestore concept

[@MC-db5gp]

2:20 Firestore would redo index when created a new document, make a change and so on, and thus a limit on field number is necessary? I cant get it :'(

[@Transmediaproducer]

I have the answer and your back in I promise! Dickens and all the rest.

[@aashishranjansingh4124]

Now we need Todd from 2022

[@goransvensson8816]

good, but I would like to have it in text instead.

[@marcelomms4021]

If I want to store user's tenats, should I create an array of references?

[@zensic]

Thank you so much!

[@shashikantdurge]

#askfirebase Are arrays ordered as they are sent? Suppose I write [ b , a , c] are they stored in the same order or [ a , b , c ] ?

[@FikirLyrics]

.... where i like to called getting uninstall ... lmao

[@ManishPatel3030]

It's very optimize way to store data in collection and send/receive in JSON.