What is a DynamoDB GSI (Global Secondary Index) ?

[@donghuang861]

This channel deserves to be known by more people.

[@BeABetterDev]

Thank you so much for your kind words!

[@grabteawithme2560]

Exactly

[@KishanKumar-su2jp]

You Always talk about what developers really needs to understand thanks man.

[@shreyatiwari7737]

I clearly understood GSI after watching this video. Thanks for a perfect explanation.

[@BeABetterDev]

Most welcome!

[@alvaromartin6301]

This is the most simple and clear explanation I've found. Thanks.

[@BeABetterDev]

Glad you enjoyed!

[@satya4866]

One stop shop for AWS. THIS IS PROBABLY THE BEST CHANNEL I HAVE COME ACROSS..

[@BeABetterDev]

Thanks so much Satya! Welcome to the channel!

[@brittche6893]

You're hands-down the best instructor on KZbin. Even complex topics are simplified and your videos are concise and straight to the point. Thank you good sir!

[@SilverEagle247]

I have read so many articles, watched so many videos on DynamoDB but none of them come close to the clarity of your explanation. U got yourself a new subscriber. Great job!

[@BeABetterDev]

Thank you so much Silver. You're kind words really mean a lot to me :)

[@haris1558ssss]

I finally understood what GSIs are. Thank you so much, perfect explanation.

[@BeABetterDev]

Great to hear!

[@abdulmuyeez]

Very helpful while i was creating my terraform script to understand whether do i need GSI or not. Subscribed!

[@mejiger]

Best tutorials on dynamodb

[@BeABetterDev]

Thanks Mej!

[@joshuapasaba1961]

This is one of those tutorial videos where you just KNOW that you should immediately subscribe. This channel is a huge help for me a beginner and I look forward to watching more of your videos.

[@shashankdahake1453]

There are lots of confusion and misconceptions were there, all are cleared with your one video ... Thumb up ... Thanks ...

[@AjitGoel]

One of the best explanations(video, audio, text) that I have seen or read. Thank you so much @BeABetterDev.

[@BeABetterDev]

Thank you so much for your kind words Ajit!

[@justsomeguy1408]

Really appreciate your content. I was working on a DynamoDB project with a friend a while back and we were struggling with how to handle some of these queries. We always check documentation, but videos like yours are extremely helpful for making it plain and clear. The visuals are particularly helpful.

[@BeABetterDev]

Thanks so much for the kind words Alec! I'm so glad these videos are helping out so many. Cheers, Daniel

[@juhisharma3334]

Wow I am binge watching your videos. Tech made simple and fun. Subscribed!!

[@nitesh2k6]

I was totally confused about GSI Before watching your video, but after watching my concept got clear. Thanks a lot for sharing the information in very simplified manner

[@BeABetterDev]

Happy to help!

[@jatinjindalj]

​@@BeABetterDev Very well explained and covering a good ground as well. One thing I am still confused about is that can we query on a partition key and an attribute from GSI in 1 query? I mean without resulting in a scan of table etc? e.g lets say the query is give me all the USA accounts which were created in last month? Will this query hit just the GSI or GSI + Primary key index?

[@CptSupermrkt]

Wow. I watch a lot of tech, cloud, Linux, etc. videos, and honestly this is one of the best videos I've seen. Clear, concise, to the point. Subscribed!

[@BeABetterDev]

Thank you so much for your kind words!

[@srogirala]

This is the best explanation of GSI

[@BeABetterDev]

Thanks so much for the support!

[@yogithakakarla1716]

Much clearly explained. Have already started visiting all ur videos on Playlist

[@BeABetterDev]

Thanks so much Yogitha. If you have any recommendations of videos you'd like to see do let me know!

[@Ko0lHaNDLuKex]

This clarified a lot of misconceptions I had around GSIs. Many thanks!

[@BeABetterDev]

Very welcome!

[@satya4866]

The author explains the concepts in a very nice way... Awesome thanks for posting nice stuff..

[@BeABetterDev]

Thank you Satya!

[@simranbanwait]

Great video... You can make these videos again and again every year as an update.

[@RovshanMusayev]

Your all DynamoDB videos are informative and just hits to target of what needs to know

[@BeABetterDev]

Thanks so much Rovshan! I try to to cover the most important topics. Keep a look out for more DynamoDB videos coming soon!

[@D3r3NB]

It feels great to grasp everything in a video! obviously due to simple and clear explanations!

[@BeABetterDev]

Great to hear!

[@MasoudGhaffarinia]

That is a great video. Thank you. One thing you might want to fix: you mentioned it is the same as having another partition key, however it would be better if you also mention that gsi does not need to be unique.

[@AdityaKumar-jj9cq]

Thank you! for mentioning this

[@rajatagarwal2379]

This makes it so easy to clearly understand GSI and LSI !! Keep up the good work.

[@BeABetterDev]

Thanks, will do!

[@reddington7364]

I have never seen a video with 0 Dislikes. Loved the video and very informative. Thank you so much.

[@BeABetterDev]

Glad you liked it!

[@tapiwanashemagwati8698]

@@BeABetterDev Usually never comment on vids but your content is awesome, keep it up and if you create a paid course let us know!

[@jackakif]

smooth explanation ,surely will be checking your videos first if i m confused about anything

[@augusto2581]

Awesome explanation, thank you very much.

[@fitnecessity7386]

Thank you for making GSI so easy to understand!

[@BeABetterDev]

Happy to help!

[@zPiranhaz]

Glad that I found your channel! Good and concise explanations for DynamoDB which saves a lot of reading of the rather exhausting documentation.

[@BeABetterDev]

Thanks zPiranhaz! Your comment is exactly why I decided to start this channel - I find the existing AWS documentation makes it difficult to understand whats important, and what you can gloss over. I hope to continue making videos like this one highlighting the important concepts. Thanks again for the support.

[@tanvir9982000]

Wonderfully explained! Thanks a lot

[@BeABetterDev]

You're very welcome!

[@SimoneIovane]

I am a "Better Dev" with your channel 🙃

[@BeABetterDev]

I'm glad I can help Simone!

[@komal1090]

Wow, this is one of my best channel for learning tech. Can you please write a query to get data from GSIs using python code?

[@BeABetterDev]

Thanks so much for your kind words Komal!

[@renancirillo3580]

Great explanation! much better than the course I've bought. Tks a lot!!

[@BeABetterDev]

Thanks Renan!

[@JesusAlfredoHernandezOrozco]

Great video! It explains clearly the concepts involved. I recomend it.

[@shoukimichaelsouri7251]

Excellent video indeed - well presented. It would be great if you would please have a video on Local secondary index and how it compares to the GSI. In addition, the concept of parallel scan and how it will remediate scan operation, etc. Well done!

[@BeABetterDev]

Hi Shouki, Taking into account your suggestion, I recently released a video on LSIs. You can check it out here: kzbin.info/www/bejne/j2nKfqKQhLKMr8k I'm definitely going to be doing a video on parallel scans in the future. Look forward to it soon!

[@praneeth0820]

At last found answers to my questions .... great explanation 👏👏

[@BeABetterDev]

Glad it was helpful!

[@stormingbarney4498]

Excellent explanation - thanks and keep up the good work!

[@BeABetterDev]

Thanks so much Storming Barney!

[@learnbatta]

awesome!!! thanks! Really appreciate your content.

[@nathanielkswas]

Stellar explanation, thank you _so_ much 🙏

[@ChrisMcFlyDude]

Excellent explanation. Cheers.

[@AnimeshPaul23]

Great explanation. I do have a question though. At 5:38 what happens when another record with originCountry = Germany is inserted? How is it stored in the OriginCountry GSI Table (with OriginCountry being the partitiionKey)? Does the index table store multiple records having the same partitionKey?

[@shaleenmundra8319]

Great explanation.

[@BeABetterDev]

Thanks Shaleen!

[@RamaKrishnaReddy-ff6wh]

the videos are really helpful, topics are really explained well, subscribed, thnx

[@BeABetterDev]

Glad you like them!

[@jumboliah13]

fantastic content and delivery, thanks!

[@AVISH747]

This is exceptional. Can't thank you enough!

[@BeABetterDev]

You're very welcome!

[@shubhoshaha9219]

I loved your explanation... its so intuitive and informative. Subscribed :)

[@BeABetterDev]

Thanks Shubho and welcome!

[@dashmasterful]

Amazing explanation bro.

[@thirue8237]

Great explanation

[@asifbalasinor6695]

This really cleared a lot of things. Thanks.

[@BeABetterDev]

Glad it was helpful!

[@likhitag3771]

Great content. Please keep up the awesome work 👍🏻

[@BeABetterDev]

Thank you Likhita!

[@XinWongDigital]

great video. If we use country as partition key that is not in a randam form. How do we garantee uniform distribution to our gsi table?

[@jriverox]

This channel is awsome

[@BeABetterDev]

Thanks so much Jhony!

[@aditisharma8437]

Thank you, this was super useful!

[@huyletran1221]

very helpful. Thank so much!

[@BeABetterDev]

You're welcome!

[@MrKiraBR]

Awesome, thanks!

[@tonykirimi9998]

Thanks a lot for this wonderful content

[@BeABetterDev]

Thanks for the support tony!

[@michalis2463]

Great video man! Keep up the good stuff!

[@BeABetterDev]

Thank you Michal! I am very glad you enjoyed

[@evawong5283]

best dynamo video

[@gshan994]

Thank you so much.

[@JagjitBrawler]

Any insights on the advantage of using something like DynamoDB (which I'd consider more of a document/key-value hybrid, where you essentially have to create GSI's and stuff to query based on attributes) vs. MongoDB, which doesn't have the same restrictions? I guess I'm trying to understand a use case in which DynamoDB would be superior....

[@omphemetsemafoko830]

Thank you for the explanation.

[@BeABetterDev]

Glad it was helpful!

[@lioneld3149]

Nice video man, keep going, very helpfull

[@BeABetterDev]

Thank you! I appreciate the support.

[@boxers999]

Awesome, thanks I’ve subscribed.

[@BeABetterDev]

Welcome!

[@flippingWebsites]

thanks! very helpful

[@BeABetterDev]

You're welcome!

[@tony098uk]

great explination

[@TomerBenDavid]

Excellent!

[@sourishdutta9600]

Really great one!! Pro 👍👌

[@BeABetterDev]

Thank you! Cheers!

[@GauravKumar-rh1yl]

Very nice video

[@BeABetterDev]

Thanks!

[@toto_frs620]

Thanks for the video. Creating a GSI will occupy double the space in the database?

[@pengli7213]

As mentioned in the video, it's better to use uniformed distributed key as GSI. The example uses country, I guess that is not a good choice?

[@BeABetterDev]

Hi Peng! Great point. Yes this example was more for demonstration purposes. Glad you caught on!

[@patpatpat4393]

@@BeABetterDev Hi! Thank you for answering this. I have the same concern as Peng mentioned. And, in this case, if I really need to have a GSI that allows me query on county, and sort the results by the lastUpdatedAt, what should I do?

[@jamescarroll489]

Good stuff! Thanks for the content! Cost is often overlooked in these design patterns. It basically doubles your storage, correct?

[@yagok2583]

This video was so good! You deserve more subs! A quick question though: I noticed that having a GSI charges you, even if you are under the Free Tier. Since I'm early on the development phase, I did not want to be paying for it. Should I just create another table and maintain it myself instead of making use of the GSIs?

[@BeABetterDev]

Hi Yago, you can indeed set up a separate table as an alternative until you are ready to set up your GSI.

[@yagok2583]

@@BeABetterDev thank you so muuch

[@BeABetterDev]

You're very welcome!

[@sakshichahal9950]

Great video. Really helped. Thank you so much for your efforts. :)

[@BeABetterDev]

Thanks Sakshi! Glad you enjoyed.

[@amanpreetsingh6257]

Question on Naive approach, "FilterExpression" is applied after DynamoDb has completed "Scan" operation right ?

[@BeABetterDev]

correct!

[@meoysnhaet]

Can you please explain we can use GSI for implementing Priority Queue ?

[@matc8085]

As always love your vids. what does this mean: "GSI partition key requires uniform data distribution"

[@BeABetterDev]

Hi Matt, What I mean is that ideally the values of GSI should be unique - like a UUID, and not categorical, i.e. ORDERED, SHIPPED, DELIVERED. The latter will cause all records with the same value to be located on the same partition. Keep in mind there is a limited amount of throughput you can achieve per partition, so this may become a problem for some applications.

[@vermoidvermoid7124]

Any common design patterns you can recommend for the staleness issue. That would be really appreciated

[@rohitkochar9022]

Amazing talk as always. Love your videos and I have learnt a lot. One quick qs. I created a GSI on one of the date column but since I have to do a between , I had to use scan operation. Is this a bad way to get data? I can use index with query but that won't let me do any between. So is table scan on gsi an expensive operation?

[@SeleshiYirga]

thanks , this is really good.

[@BeABetterDev]

Thanks for the support!

[@blameItleaveit]

Local Secondary Indexes or Global Secondary Index Local Secondary Indexes still rely on the original Hash Key. When you supply a table with hash+range, think about the LSI as hash+range1, hash+range2.. hash+range6. You get 5 more range attributes to query on. Also, there is only one provisioned throughput. Global Secondary Indexes defines a new paradigm - different hash/range keys per index. This breaks the original usage of one hash key per table. This is also why when defining GSI you are required to add a provisioned throughput per index and pay for it.

[@gnaneshwarrao174]

Race condition sounds scary. I would like to know more about them. I am sure there will be some best practices out there to properly use the API in order to not fall into it.

[@jonathanhle]

So good. How do I send you a tip. Great content.

[@BeABetterDev]

Thanks so much Jonathan for your kind words! If you would like to make any contribution, I have a patreon account here: www.patreon.com/awssimplified . Thanks again and have a great week!

[@jonathanhle]

@@BeABetterDev I'm a patron! Thanks for putting out such great content.

[@BeABetterDev]

Thanks Jonathan, really appreciate the support!

[@caof2005]

You mentioned that it is allowed only 20 GSI, is that by table or by DB?

[@BeABetterDev]

Hi Carlos, thats per table.

[@bluetoba]

So, can you update the origincountry field itself? For example, you already had a GSI on origincountry and decided to update "Germany" to "Italy". Will it also update the GSI's partition key from Germany to Italy?

[@martinzen]

THANK YOU 🙏🏻

[@BeABetterDev]

You are very welcome!

[@agytjax]

This is exactly the problem with DynamoDB (or any NoSQL DB for that matter). Creating GSI's for every new access patterns (once your main table is created) is a big maintenance hassle for projects. In RDBMS, you create indices (and not separate tables) to query on non-primary key columns only if we observe performance issues. I understand that is done to support hyper-scale, but I wanted to point this out. Finally, its all about trade-offs and DynamoDB is by no means a silver bullet. It is a call that respective project leads and architects to have to take based on what they are willing to trade off. Let's not write off RDBMS as yet !

[@jurilunin7600]

In case of using GraphQL, to avoid race conditions, probably, making separate table for each entity type is better than making GSI for each entity type.

[@getvasued]

This video makes it sound like GSI's sort key has to be the same as that of the primary key, while in reality a GSI's sort key can be an entirely different attribute (in fact, a GSI need not necessarily be a composite primary key)

[@regmanua]

Very useful video, thanks! But I have one question about queries by GSI. Should we always use an IndexName option in query request, if we are looking for specific value of this GSI? I thought that KeyConditionExpression is enough, but my teammate says that IndexName must be present with value of this GSI name....

[@jivanmainali1742]

should we not have different table for different collections???

[@VenkatKarthickKaron]

If i have to create a Global Secondary Index for a map type of data or Nested String Data how to do so?

[@SwapanD]

Need a clarity - as it is allowing up to 20 more secondary tables for a primary table, will it not consume additional space? Is that not a drawback?

[@BeABetterDev]

Hi Swapan. It is definitely a drawback and can affect the amount dynamo ends up costing.

[@SergiiStarodubtsev]

I guess there no automatic sync s when removing or updating data from primary tables and the updating gsi one?

[@BeABetterDev]

Hi Sergey, the sync happens automatically for you behind the scenes.

[@jamestufarelli7888]

Hey, this is a great video. I have a use case that I am having a hard time modeling with dynamoDB. I have an application that contains a list of items in your area (lets say it scales up to millions), and each user will be shown a list of items, but that item can only be shown to them ONCE. So after a user has seen an item, he/she shouldnt be shown it again. So I need to be able to query for all items in my area that the user has not seen. DynamoDB is good for query against values I know about, but what about unknown values? I thought about using Bloom filters to accomplish this, but I'm not sure if this is a good use case for dynamoDB. I am trying to avoid using a scan and filter expression since that would be costly. Thanks for your help.

[@BeABetterDev]

Hey James Thanks for the question. Bloom filter sounds like a good idea at first glance. However, it doesn't really help you too much since when you're pulling items in your area, you would need to pull the items first, followed by attempting to negate each one from your result set using your bloom filter in order to determine which ones are candidates to show. For instance, if when querying areas, you pull 10 records, you need to run your bloom filter on each one (possibly 1 by 1) to figure out if you show all 10, a subset of the 10, or none if they have all already been seen. Thinking about this a bit more, you definitely will need to keep track of which records each user has seen. I don't think there's any way around that based on your use case. However there is a question of how you store this information for optimal lookups. The two options are a) 1 record for each customer, and a column that contains a list that indicates each one they have already seen (bad idea), or b) 1 record for each customer + area they have seen combination. Based on this, I don't think Dynamo is the best option for your use case since you will likely need to do double queries (1 for regions, 1 to determine view candidates). This will likely incur some significant costs as your dataset grows. If this application does not have scaling concerns, I would suggest using a RDS based solution such as AWS aurora + some criteria based queries. If scaling is a concern, you should look into using AWS elastic search. Elasticsearch is excellent in performing "fuzzy searches" like you described and allows you to offload much of the logic you need to perform onto the data store itself. Additionally, it is horizontally scalable so dataset growth shouldnt be too much of a concern. Hope this was helpful.

[@jamestufarelli7888]

@@BeABetterDev Yes, this absolutely is helpful. Thank you! I really enjoy your videos. Please keep it up.

[@BeABetterDev]

You are very welcome and thank you for the support!

[@ankitgarg568]

I had a doubt. I read that GSIs do not support strong consistent reads, does that mean, we can't create indexes in dynamoDB (for it to support strong consistency)?

[@BeABetterDev]

Hi Ankit. You are correct, GSI's do not support consistent reads.

[@XStrikeSuper]

Can you do a video about sparse index?

[@BeABetterDev]

I'll look into Sparse Indexing for future videos. Thanks for the suggestion!

[@Pavanadiraju]

Hi , i have one question is there any way to implement one -many or many to many in Dynamo DB .

[@MyRonniej]

subscribed

[@BeABetterDev]

Thank you and welcome!