Amazon DynamoDB: The architecture that scales to 10 TRILLION requests daily

[@gkcs]

EDIT: Amazon supports 20 GSIs instead of the 4 mentioned in the video. Thanks to anand_ammathil who pointed this in the comments 😁 Glad to see you here again! If you are looking for more system design content, try my course at InterviewReady: interviewready.io/learn/system-design-course Cheers :D

[@priyanshrawat442]

I don't really understand most of the stuff you say because I am just an undergrad but all this stuff is so fascinating....I love the engineering involved in these big applications!

[@gkcs]

Cheers!

[@kalpakHere]

Great to see you back in action. Please follow it up with Spanner, Cosmos DB, Aurora, and the latest AWS' DSQL. It would be of immense help.

[@gkcs]

Thanks! Do check out my playlist on research paper breakdowns 😁

[@chandrabhanurastogi9554]

I recently came across your small videos (reels). I am loving them. Apparently that's how much time and content one needs to start a conversation.

[@gkcs]

Glad you like them!

[@Md_sadiq_Md]

10:55 💰💰 You're a good story teller besides an excellent system design teacher

[@gkcs]

Thank you :D

[@NikhilKumar-oy7mx]

Wow 😮, i used this every day in a project and fetched saved consistently but today i understood the depth 😢 Thanks a lot sirji ❤ Its a gem of a channel

[@gkcs]

Thank you!

[@pawanrajypr]

The way you walkthrough makes it easier ❤

[@harvendrasinghrathore2848]

DynamoDb also works on pay on demamd model which cost you for the RCUs and WCUs you made, I think which is great where i don't need to bother what capacity i had to choose and irregular traffic on system.

[@gkcs]

Yes that's the option they recommend :)

[@maxivy]

This guy is the future of teaching SWE online imo, very robust against AI swarming everything

[@archiliusfowl3701]

Thanks for this. It would also be interesting to see how other competing NoSQL datastores are architected.

[@gkcs]

Do checkout the research paper breakdown playlist on this channel, where we have analysed Facebook's graph and Google's timeseries databases 😁

[@anand_ammathil]

14:15 you can have upto 20 GSIs

[@gkcs]

I just checked and you are right. Thank for reporting this. I'll make an edit in the pinned comment 😁

[@rohangaonkar8912]

Few of the issues i had when working with dynamodb is how thier pricing works. They charge based on number of records scanned or written. I had a requirement where my application performs a lot of reads. Now, the problem is dynamo charges based on number of records scanned and not based on number of records matched. As long as you query based on primary key you are fine but the moment you query based on non primary attribute you are screwed, in this case dynamo essentially scans the entire table and consume a table worth of read units.

[@NoName-lz6bc]

10:57 at this point why to keep a cache at all?

[@RajKumarSingh-wi6se]

Amazing, thanks for your effort.

[@gkcs]

Thanks :D

[@harvendrasinghrathore2848]

I think a single table can allow more than 4 GSIs, beacuse i had used 6 in one of application which works on single table design. Thanks for this great content @Gaurav ❤

[@gkcs]

Yes it can allow upto 20. I didn't remember the number correctly, so there is a correction in the pinned comment :)

[@Abhigyan_Bafna]

Hey GKCS, I'm a newbie to sys designs and usually things just bounce off my head or I need to watch them multiple times to gain some sort of assumed sense but the data scheme part of this video was great! I understood almost everything and was enjoying it a lot. I dunno if you did anything different for that section or it was just the visuals, but maybe try doing whatever you did more?

[@gkcs]

Thanks, cheers :D

[@sagardarekar6303]

This is awesome!

[@suyashnigam63]

Hey Gaurav, i got confused in caching strategy, you meant when requests rate go down cache will make sure to keep on querying the db so that when there is again increase in traffic it will be ready to handle that much load through cache only ?

[@parassharma7041]

Great video as always, currently I'm reading a book "Design data intensive applications " I found it really interesting. Can you suggest some more Programing books for distributed systems and backend engineering

[@NikhilKumar-oy7mx]

Why does almost every service has some sort of usage of s3

[@gkcs]

It's a file/object store, fundamental to distributed systems. Google uses Colossus which has a similar function, and is used by nearly every system in their stack.

[@imerence6290]

How is this compared to google spanner ?

[@gkcs]

Google spanner is a consistent datastore that uses timestamps for external consistency. It's also used to store data, but the internal architecture is different.

[@simarpreetsingh7235]

Hey Gaurav, can you share where can someone find Research Papers related to Designing Systems

[@gkcs]

interviewready.io/resources Click the checkbox for staff engineer.

[@B-Billy]

@Gaurav sir, someone recently said in his video "People talk about consistenct hashing, and say we can create a Ring and blah blah blah, but nobody knows how to implement one, or how it has already been implemented". Can you please please please make a video on this topic and request you to cover any curently availabe impelementation.

[@gkcs]

Try InterviewReady's playlist video here: interviewready.io/learn/system-design-course/building-an-ecommerce-app-1-to-1m/6-database-like-memory-cache-like-recall?tab=chapters

[@vinuprasad6846]

KZbin playlists from top engineers (e.g., Gaurav Sen). --- chatgpt suggested your name when i asked for system design tips.. 😃

[@gkcs]

Cheers :D

[@Tanmay-m1i]

bro i didn't understand anything due to my poor fundamental knowledge so can you please tell me what all things i need to learn to make my fundamentals strong and to become a better engineer. I really want to understand these things so that in future i can also come up with smart ideas.

[@gkcs]

No worries, try this: kzbin.info/aero/PLMCXHnjXnTnvo6alSjVkgxV-VH6EPyvoX This is the best begineer's playlist: interviewready.io/learn/system-design-course/building-an-ecommerce-app-1-to-1m/1-what-is-system-design

[@Tanmay-m1i]

@gkcs thanks bro 👍

[@ManishKumar-qe8wt]

How to be like you? Plz suggest.

[@saivamshikrishna6729]

Hey Gaurav ! Can you make a video about snowflake architecture and it is different from others

[@gkcs]

Have a look at the Google Dremel video on this channel, which is the backbone of Google Bigquery. Snowflake works for similar usecases. You can also have a look at the Apache Spark video to get an idea of different data processing systems.

[@thePribs]

Thanks for helping the SWE community

[@gkcs]

Always!

[@HardyBardy]

Please fix my paid subscription

[@gkcs]

Hi, thanks for reporting this. Please connect with us at [e-mail] or on WhatsApp at 9920533010, we'll have this resolved as soon as possible.

[@HardyBardy]

@ this is getting bounced with wrong domain name.