AWS DynamoDB Schema Design | How to choose the right key

[@dbejjam]

Am a veteran RDBMS guy and new to NoSQL. I got to say, the way your describe NoSQL stuff is just phenomenal.

[@BeABetterDev]

Thank you so much for the kind words!

[@venkatesh6384]

This is one of the most useful video I found on dynamodb keys. You made me your subscriber

[@BeABetterDev]

Thank you for the kind words. Its comments like these that motivate me to keep making more videos. Thanks again and take care.

[@leetsai6220]

Great job explaining the topic and exploring the limitations. The way you explain a complicated subject with simplicity without sacrificing knowledge is sheer elegance. Keep it up.

[@BeABetterDev]

Thanks so much Lee! Your comments means a lot to me :)

[@tyronefrielinghaus3467]

Yes BABDev: simplitivity without sacrificing knowledge...spot on! I like the way you explain everything. When I'm learning from a tutorial I HATE having to spend half the time googling what was mentionef but not explained....thanks for NOTsoing this...it really helps. Ps...great voice too...clear&well modulated. Just easy to listen to.

[@suhasbadhe]

This is by far the best video I found for DynamoDB Keys!

[@fahimuel]

Best Dynamo DB Schema Tutorials with precision knowledge providing all the use cases. Many Thanks.

[@BeABetterDev]

Thank you so much for your kind words!

[@Xhez2slash]

Splitting up the RCU to depict the partition capacity is by far the best way I seen hot partition be explained.

[@BeABetterDev]

Thank you!

[@Xhez2slash]

@@BeABetterDev Thanks to you I got my Dev Associate!!!

[@haris1558ssss]

You're such a gifted teacher. It's not easy to grasp all this information from the documentation. Thanks for the amazing video.

[@BeABetterDev]

Thank you so much for your kind words. I actually kind of enjoy reading documentation and I'm glad I can translate that into easy to understand examples for all of my viewers. Thanks again!

[@osamaa.h.altameemi5592]

Man you really simplified it. The original documentation will take you in circles all over the place. Fantastic video.

[@BeABetterDev]

Thank you so much! I'm glad you enjoyed the video and helped you understand the concepts. Daniel

[@ChauDuong1982]

Best DynamoDb explained ever.

[@andrestone]

The whole hot partition concern and the WCU / RCU hard limits are pretty much a thing of the past. DynamoDB will automatically reshard your partitions behind the scenes down to a single item if the disproportional velocity access is there. It means you'll only reach hard limit throttling if you have a single item being responsible for more than 1000 WCU / 3000 RCU.

[@sumaiyaahmed4948]

Thanks for explaining in a simplified way. Very useful

[@BeABetterDev]

You're very welcome sumaiya!

[@AIAnimationStudio]

4 years alter... this run through is still very helpful. 👍

[@siddheshlatkar9839]

Very nicely explained.

[@BeABetterDev]

Thank you! 🙂

[@MrMazharsamo]

Simple and to the point explanation...Keep up the good work

[@BeABetterDev]

Thanks a lot!

[@srb1855]

This was excellent - wish I had watched it 6 months ago! I was actually searching for "cardinality" and your vid came up among the first results.... Thanx again.👍

[@atjams]

Fantastic video! Easy to learn. Thank you 🙏

[@codebreakthrough]

Great stuff! Very helpful, thanks.

[@vijaygokarn]

you deserve more subscribers .. keep up good work

[@BeABetterDev]

I appreciate that!

[@syedzainrehan]

Very well explained. Thank you!

[@BeABetterDev]

You're very welcome Zain!

[@gustavoangelochannel]

Awesome video, keep the good work!

[@BeABetterDev]

Thanks Gustavo! Really appreciate the kind words.

[@haciendadad]

Excellent video. Good speaker and provides content very well! Easy to learn from!!

[@BeABetterDev]

Much appreciated!

[@sureshgandhi3918]

You are awesome and one of the best teachers I have ever seen. You boil down the concepts into simpler concepts extremely well. API Gateway tutorials please (I can even pay you for that :P)

[@BeABetterDev]

Thanks Suresh for such kind words. It is comments like yours that motivate to keep making content. Thank you!

[@blue_moon1_1]

Impressed.... Very informative

[@BeABetterDev]

Glad you liked it!

[@marcoavendano6593]

I agree with all the comments here. Nice work describing this in simple terms.

[@BeABetterDev]

Thank you!

[@vimalneha]

Very well explained!

[@BeABetterDev]

Thanks vimal!

[@ashguru]

very well explained.. all stuff no fluff.

[@BeABetterDev]

Thanks ashutosh!

[@akashseth7452]

You just won a subscriber. High-quality stuff (y)

[@BeABetterDev]

Thanks Akash! Glad you enjoyed.

[@sorover111]

These are really good mane, keep it up 👍🏻

[@BeABetterDev]

Thanks Michael!

[@HamzaAli-hv4md]

Great video. Thank you so much!

[@BeABetterDev]

Thank Hamza, super glad it was helpful!

[@ishaanme91]

great explanation!

[@BeABetterDev]

Thanks Ishaan!

[@georgesoniarachel]

Awesome tutorial !!

[@BeABetterDev]

Thank you! Cheers!

[@ramamoorthyvasanth]

This is super useful, please create a video on local secondary index as well

[@BeABetterDev]

Thanks Rama! I'll add LSIs to my backlog of ideas. Cheers.

[@BeABetterDev]

Hi Rama, in response to your feedback, I just created a video on Local Secondary Indexes - check it out here: kzbin.info/www/bejne/j2nKfqKQhLKMr8k

[@sagarsinghrajpoot6788]

Very Nice Explanation :)

[@BeABetterDev]

Thank you Sagar!

[@aliciacollymore220]

This was super useful, would be awesome if you could do a video on GSI and how that works, can't seem to wrap my head around it fully

[@BeABetterDev]

Thanks Alicia. You''re feedback means a lot. I have one coming out this weekend on DynamoDB Autoscaling, but definitely plan on doing one on GSIs in the next month or so. Thanks again!

[@BeABetterDev]

Hey alicia, check out this new video I just posted on GSIs! kzbin.info/www/bejne/n5mwgJ-YbaqNgac

[@fitnecessity7386]

Nicely explained! Subscribed :)

[@BeABetterDev]

Awesome, thank you!

[@xiangyuncui373]

Very clear, easy to understand, thank you!

[@BeABetterDev]

Glad it was helpful!

[@bhageerathbabuprathi1043]

Excellent video . It really helps. Thanks a lot.

[@BeABetterDev]

Glad it helped!

[@mangeshxjoshi]

Excellent video on DynamoDb partition key concepts , it would really help. If you could provide any video on DAX whenit can be used and its pros and cons

[@BeABetterDev]

Thank you for the kind words. A video on DAX is coming soon!

[@yogithakakarla1716]

Most useful video

[@BeABetterDev]

Thanks Yogitha!

[@rayprusia4753]

Great video Thanks

[@BeABetterDev]

Thanks Ray!

[@griscz]

Very helpful thank you!

[@BeABetterDev]

You're very welcome!

[@vijayaprasadramachandran7603]

Awesome .. Simply put

[@BeABetterDev]

Thank you!

[@kunalvids]

Thanks for wonderful video

[@BeABetterDev]

You're very welcome kunal!

[@_cipriangg_]

Thank you for the info!

[@XxGabberlordxX]

Hey! Your videos are very helpful and I learn a lot of things! For me and maybe another people it might be interesting to see an end to end project. You don't have to implement it but something like a general concept would be awesome. For example something like: Deploy a website and a REST service via AWS. Which services should we use and how to secure the backend and how to connect the REST API to a DB. Things like this would be absolutely awesome :) I'll definitely follow you and watch the other tutorials. Cheers mate :)

[@BeABetterDev]

Thanks Neal! An End to End video has been on my radar for a while. Thanks for the suggestion!

[@XxGabberlordxX]

@@BeABetterDev An additional suggestion that might be easier and also makes a lot in for example the whole aws rds context is a tutorial about bastions/jump boxes. I. e. host an RDS Aurora db in a private vpc and then you create a jumbox to connect to this db via SSH. Have a great day :)

[@PulsePTYork]

How do you not have more subs? your channel is great! Keep it up mate!

[@BeABetterDev]

Thanks a ton!

[@anup00717]

Very useful.. thanks for the video, AWS docs are jargon :P

[@BeABetterDev]

Glad it helped!

[@pedrouriarterodriguez899]

Excellent video and information, keep doing such quality content

[@BeABetterDev]

Thanks you so much!

[@jeffryjohn2508]

Thank you for this tutorial 🙏🙏🙏

[@BeABetterDev]

You're so welcome Jeffry!

[@akshaykalambe9054]

Very nicely explained. Keep it up :)

[@BeABetterDev]

Thanks Akshay!

[@XinWongDigital]

Thank u for sharing. Wud b cool if u mention filtering in the video

[@BeABetterDev]

Great suggestion!

[@jeffinj7707]

Great content

[@BeABetterDev]

Thanks Jeffin!

[@sameerahmadbalaganur8958]

Really good

[@BeABetterDev]

Thanks!

[@emmanueladetutu8739]

Awesome!! Thanks for this. It was very helpfull.

[@BeABetterDev]

You're welcome!

[@rakeshmishra8208]

Thanks, it's a great video :)

[@BeABetterDev]

Glad you liked it!

[@wajay2006]

Excellent video !! Do you have any video on data modelling a relational database in DynamoDB

[@BeABetterDev]

Hi Ajay, not yet but thanks for the great tip!

[@calvinedwards6338]

wow only 1.93K subs.. what a crime. Subbed!

[@BeABetterDev]

Thank you so much Calvin for your kind words!

[@criticalflow8733]

Nice one !! i just subscribed

[@BeABetterDev]

Thank you!

[@MiralkumarPatel]

Speaking of other strategies: How do you create composite primary key with 3+ columns?

[@BeABetterDev]

Hi Miral, I think it would be as simple as just appending each value using a delimiter such as value1-value2-value3. Keep in mind though this pattern requires you to know all these values before querying in order to find your records in Dynamo. Hope this helps, Daniel

[@MiralkumarPatel]

@@BeABetterDev Thank for reply. That's the catch, the third column value is a timestamp and for that reason, can't perform BatchGetItem.

[@rohitkochar9022]

also, with partiQL you can query table with a value without cretaing a GSI on that column. is that a new feature?

[@BeABetterDev]

Hi Rohit, the thing you need to be careful about using PartiQL is that performing queries are non-indexed columns can result in a scan operation instead of a query. This can cause high costs and poor performance for these types of operations. Hope this helps clarify.

[@rishabhkohli7170]

Thanks amazing video

[@BeABetterDev]

Thank you !

[@kyriacoslouca2799]

Very nicely explained indeed. I have seen some examples where the Partition Key is an UUID value. However, in some cases, I need my primary key to be an auto-increment integer like the one many SQL databases provide. What is your suggestion?

[@BeABetterDev]

Hi Kyriacos, Unfortunately auto-increment isnt supported in DynamoDB. Generally speaking, in a NoSQL database it is advised to use a meaningful primary key so you can quickly retrieve your data by said key. Your access pattern should define what your primary key should be (even a composite key could be used). Your requirements make me think a traditional relational database could be better suited for your use case.

[@shivamvats6146]

7:03 Why partion 3 row is in Partition 1 ? Shouldn't it be under a new partition 3 ? Please help, this is my only doubt left .

[@oldoctopus393]

Double this question

[@BeABetterDev]

The location the partition lands on is independent on the data. Rows that have differnt partition keys can land on the same partition.

[@IanHuang-ch1cn]

Sorry, I would like to ask a question. At 6:18, you said that item with the partition key 3 will be stored in the same partition as item with partition key 1. But aren't they having different hash value after you put their partition key into the hash function? So why did they end up in the same partition?

[@BeABetterDev]

Hi 黃翌軒, Depending on the size of your table, DynamoDB may store values with a different partition key on the same physical partition. As your data size or throughput demands grow, DynamoDB can re-split these partitions into smaller ones. Hope this helps.

[@Fernando1989mg]

nice video, thanks u!!!

[@BeABetterDev]

Glad you liked it!

[@kshitijvats5261]

So does querying with parition and sort key will take logn time? Partition key will determine with hashing function the correct partition and then in that partition we will use sort key(log n time, where n is rows with same partition key value) to get to the reuqired row?

[@rohitkochar9022]

How did you define a date as sort key? I do not see type date. Do you define date as string?

[@BeABetterDev]

Hi Rohit date is just a string in this

[@hetalraval3928]

In my case i want to scan all data each time to compare with my response to know where update occur or insert and according to that i want to perform other requests...so what could be the best possible way?

[@mohammedabdullah9537]

Hey! how to find my table's details or description on AWS DynamoDB?

[@BeABetterDev]

Hi Mohammed, this should be in the Overview tab on the console after clicking the table name.

[@niteshgupta8436]

Great video. Just one doubt. How to design my table if i have more than 2 fields for uniqueness (let's say 3 fields defining uniqueness. For example, in vehicle, make+model+year is unique)

[@venkatk161]

Thank you. Well explained. Another subscriber here :-)

[@BeABetterDev]

Thank you so much Venkat! Welcome to the channel.

[@tejasdevgekar1154]

thanks dude

[@robertfrank2345]

you rock!

[@EdwardLegaspi]

Hi. When designing a DB schema in DynamoDB how do you store the traditional lookup values like countries? In it's own table country with countryCode, countryName attributes? What's the best practice here? Thanks.

[@lardosian]

Thank you, just subbed. I'm about to get my hands dirty with AWS for the first time, use case is building an API and a PWA with React, was thinking of using the serverless framework, some advise to steer me in right direction would be greatly appreciated, I'm also doing my own research so it looks like I will need the combination of the following..API Gateway, Lamda functions, Dynamo DB, Cognito for backend, S3 for hosting, serverless for my framework, I'm sure theres more, thank you!

[@BeABetterDev]

Hi Lardosian! Welcome to the channel. Great to hear you're getting familiar with AWS. In terms of your use cases, It sounds like you've identified the right AWS technologies for your use cases. Here are some additional videos I think could be helpful. IAM - IAM permissions are the bread and butter of AWS. You should get very familiar with them - kzbin.info/www/bejne/r2nGk36AdtRmeJY Lambda Introduction - kzbin.info/www/bejne/n4asiHpmnZd3bpY Building a REST API with API Gateway + Lambda - kzbin.info/www/bejne/q3fWkpx7nddksNU Basic DynamoDB Operations - kzbin.info/aero/PL9nWRykSBSFgmcJMpr7XAk3BC0VPAK_i- Hope this helps!

[@lardosian]

@@BeABetterDev Thank you very much, apologies for the late reply.

[@crimsonderrick]

Thank you

[@BeABetterDev]

You are very welcome!

[@lardosian]

So in an amplify project for example, this schema is defined in the graphql schema yes? Do you have any such videos. I found a very good post on Dev.to from Nader Dabit about creating a robust graphql schema with directives, thanks!

[@georgemap]

Great Video, thank you for all the information! I have a question about filtering - which solution would you approach if you wanted to filter with multiple conditions? Equivalent to: WHERE country=„…“ & age=„…“ & … and so on (note: country and age are not pk or sk and without using filterExpressions) Or is there not a nice solution with dynamodb to that and aurora would be a better choice? Thank you in advance

[@ashleyalex2713]

Suppose if we design sort key as countrycode#date, how we will be able to sort date only as we have country code along with it

[@rhushikeshlokhande9108]

Very Knowledgeable video Clare my table schema confusion. Please do share the Github link for code reference for serverless application if you have any..

[@BeABetterDev]

Thank you Rhushikesh! I will post github links soon. :)

[@64kuroina]

how many items can I set in the same partition key?

[@BeABetterDev]

Hi ruru, There is no limit. However keep in mind that there is a 10gb max partition size in dynamo. I believe the partition splits afterwards. If you have too much throughput on the same partition key you make get throttling errors.

[@64kuroina]

@@BeABetterDev thanks for answering me, the video was very helpful😁👍

[@sankalpsipaulya584]

At timestamp 2:29 , shouldn't there be three partition created.Or I am getting it wrong here

[@mdhruv1]

Please can we get a tutorial. Where I can use curl to create an presigned url for s3 bucket object to load an file and download a file .

[@geoa9722]

how to use update method with combination of partition key and sort key?

[@cjj1120]

Is GSI a new feature since it’s not mentioned in this informative video

[@BeABetterDev]

Yep that sounds correct

[@alulaleak]

Is there a reason to use DAX over CloudFront??

[@surbhisingi5214]

Is it possible for you to create a video on how to create DynamoDB table - having LSI and GSIs through Cloudformation script and add items in it using put_item from boto3 python SDK?

[@gauravshetty1436]

what if partition key itself is unique an there are 1 million records example email id as partition key in user table?will there be a problem with too many partitions?

[@BeABetterDev]

Hi Gaurav, Dynamo would split your data across multiple partitions in that case. There is no limit that I am aware of regarding number of partitions.

[@Tellalca]

6:20 How does the item with Account Id = 3 goes into Partition 1? Shouldn't it go to Partition 3 since the Account Id attribute is defined as partition key?

[@rockylovesall]

IN my understanding, When the hash function is applied on the account Id it generated the target as partition 1. E.g. id % 2 ==0 will go in partition 2 and rest in 1. Hope it helps

[@andresm9051]

Great video, I got a question through an example If I had a sign up method in app and I can do it by email or phone number I was thinking about primary key because of cognito Id user, and that user sell products using many categories can have those products and categories under same table and row and query it by it's I'd, it's a good practice ? thanks

[@jean-louisgouwy]

Most of the time, we depend of business rules. I mean, recently, we create a table with just an uuid (for each object). Then later, The business would like to retrieve thoses objects but without knowing the uuid, they need to retrieve the object by the name And lastModificationDate (between two dates). How do you manage that ? You have to create some gsi... But we do not have always all the information when we start the archtecture. This is very complicated when your project evolves with anothers features that you do not anticipate. This is my pain using dynamodb over mysql before.

[@JamesSmith-cm7sg]

The use case for Dynamodb is driven by high performance, so ultimately the business needs to understand the complexity of new queries. You can also utilise something like dynamo streams to update something like elastic search or ETL the data into a relational DB for complex queries.

[@SatejPrabhu]

Good video.. after spending over a decade in SQL database.. its challenging to understand how SQL schema could be translated/redesigned to NOSQL. If you could make a video by taking example of AdventureWorks to show how you would design that use case in NOSQL, that would be great.

[@NiclasGleesborg]

How can a row with partition key "3" end up on Partition 1? 3:25

[@BeABetterDev]

Hi Niclas. Partition assignment is completely random. Hope this clarifies.

[@deladonics]

I must be missing something fundamental about Dynamo because I keep looking up how to retrieve all items in my table in a sorted order, but I keep ending up on articles and videos like this where they talk about read capacity and partitioning. I want SELECT * FROM table ORDER BY time ASC. I get the feeling I could give everything the same partition key and unique sort key and have them come back ordered but that has to be the wrong way to achieve my goal.

[@NoSQLKnowHow]

I am not sure of your exact use case and why you need to do a Select * FROM table, but I am pure guessing from the way you state your comment that more than likely you are trying to apply a relational DB mindset onto a NoSQL database, in this case DynamoDB. If you want to expand your comment, I am happy to try and help you though.

[@deladonics]

@@NoSQLKnowHow I have been messing around with Dynamo to build a serverless version of my blog. The traffic I get would be well under the free tier limit for both Lambda and Dynamo, so I could have a blog hosted for cheap. As an example of what I'm trying to do, I would call Dynamo to retrieve my blog posts and have them return in order of the sort key which is a timestamp. So far, I can only call scan and sort them after they return but eventually I'll want to do pagination and only return ten of the most recent posts.

[@BeABetterDev]

Hey Stephen! Based on your description, it sounds like you're trying to use Dynamo as if it were a traditional Relational Database. NoSql databases like Dynamo excel at quick retrieval for known values where you know the key (either singular or composite) that you are looking for. It can be extended to query for a key + a second attribute (sort key), but its functionality is pretty limited. Concerning your specific problem, you can solve this by having a table setup where your partition key is something like blog id (singular value) and your range/sort key is creation date. The combination of these values should be unique (as you can never create two blog posts at the same time). Using this approach, you can perform a paginated query in descending order on the sort key to get top X results (set your LIMIT to be X on your query). This isn't an ideal setup but should solve your problem. Hope this helps!

[@abhaygodbole9194]

Thanks for this very insightful session. Just would like to know where I can ask questions. I have one doubt about PutItem, which is not related to this video.

[@BeABetterDev]

Thanks Abhay! feel free to ask here

[@abhaygodbole9194]

I have asked it in your other video related to inserting items in DynamoDB.. Thanks

[@ThiruMurugan-d6r]

how to pagination using dynamoose

[@usamasaleem5624]

Can you please share the slides

[@jivanmainali1742]

How many partition Key could dynamodb could have??

[@BeABetterDev]

Hi Jivan, technically it is infinite.

[@rudyramadhana4127]

When you say 'sort key', I have been hearing 'sword key' instead, my bad lol

[@shamstabrez2986]

Is this tutorial valid for 2022

[@BeABetterDev]

Yes!