This one is an awesome talk. Jaso goes through important innards of DynamoDB, including secondary indexes, throttling, burst capacity, accumulating read capacity, adaptive capacity, etc.
@ahmxtb Жыл бұрын
I've read both Dynamo 2007 and DynamoDB 2022 papers and this is a great summary. Thanks.
@awssupport Жыл бұрын
Glad you enjoyed it, Ahmet! 😃 ^KR
@CAMorales823 жыл бұрын
0:00 Intro 2:39 Agenda - GetItem/PutItem - Auto Scaling - Backup Restore - Streams - Global Tables 3:04 GetItem/PutItem - Request Router - Paxos - Partition Metadata System - Tables (hashing, partitioning) - Eventual Consistency - Storage Nodes (B-tree, Replication Log) - System Management (Auto Admin, Partition Repair) - Secondary Index (Log Propagator) - Provisioning Table Capacity - Adaptive Capacity 28:48 Auto Scaling 33:54 Backup Restore - Point in Time - On-demand backup 41:43 Streams 44:45 Global Tables
@shawapratim3 ай бұрын
Amazing. Just what I needed. Thank you for making this available here.
@galeop Жыл бұрын
Key takeaways: - storage nodes acknowledge a WRITE (on a partition) back to the request router if 2 out of the 3 storage nodes for the affected partition successfully completed the WRITE. So when you perform an "eventually consistent READ", the odds to get a strongly consistent result are 2/3. -The 3 storage nodes of a partition decide between themselves which one will be the partition leader. - Leader nodes store the partition data into a B-tree index, and they also maintain a "replication log" (a kind of TX log).
@saumilkapadia884 жыл бұрын
very nicely described. kept on watching and watching. came back to this video multiple times when i understood more and more.
@VitalyZdanevich4 жыл бұрын
Thank you for dark slides.
@rahulat853 жыл бұрын
Good talk! Seems like a industry guest talking about dynamodb in classroom.
@brandonhunter30365 жыл бұрын
damn that was a good talk
@Himanshu-mb8nl7 ай бұрын
At 9:21, could the client be talking to the request router (RR) in an availability zone (AZ) and it isn't necessary that it has the leader storage node, and therefore the RR might have to send the request to the storage node leader to a different AZ. Nothing wrong with that but I wonder if there are any performance savings in having writes go directly to the RR in the AZ with the leader storage node.
@70lan72 жыл бұрын
Best talk!
@galeop Жыл бұрын
18:00 I guess we're talking about *global* 2ndary indexes here, and not *local* 2ndary indexes, correct ? Does this propagation of a new value from the main table to the 2ndary index happen asynchronously?
@awssupport Жыл бұрын
Hi Galeop, I appreciate your question and have raised this with the necessary team. Please check out this doc which may provide more insight into this: go.aws/3jETlOJ You're also welcome to reach out and discuss this matter on re:Post, by using this link: go.aws/aws-repost. ^ES
@frankren13335 жыл бұрын
Trying to guess the PID controller part, seems it only needs the P(proportion)? let multiplier = 1.5 Let's say we want to allow roughly 60 seconds of 150% consumed over provisioning. Then on each iteration(say, a second), do: multiplier = multiplier + (provisioned - consumed) / provisioned / 60 # 60 is for 60 seconds multiplier = min(1.5, max(1, multiplier)) # clamp it between 1 & 1.5 This way the consumed can surge to 1.5 times provisioned and will slowly go down to 1, then keep as 1 until consumed is below provisioned and slowly bring multiplier back to 1.5
@pemessh4 жыл бұрын
Great talk.
@kevin89184 жыл бұрын
the last few sentences are like "we are hiring; come and work with us" lol
@justmeandmy4 жыл бұрын
Yeah, and now that you've seen the talk you're already partially onboarded :P
@Mike-ci5io2 жыл бұрын
Does the replication log store the entire history of the data or just current state ?
@ahmxtb Жыл бұрын
usually the writeahead log only stores the portion of the data that's residing in the memory that's not yet flushed to disk. usually some databases that use Log-structured merge trees store most recent data in in-memory trees + writeahead log, and periodically flush to disk files.
@anjalivas11114 жыл бұрын
Nice. . . very nice.
@Tommy-dd5pz2 жыл бұрын
I think it's looks like nanoseconds but actually milliseconds? 50:42
@youran2 жыл бұрын
Where can I download the PPT, please?
@NR-bt7yz Жыл бұрын
d1.awsstatic.com/events/reinvent/2019/Amazon_DynamoDB_Under_the_hood_of_a_hyperscale_database_DAT325.pdf (it's from 2019, but appears to be identical)
@sagara93895 жыл бұрын
Dynamodb is awsome for bigdata works but it is really hard to work with CRUD operations
@dreamtheater_925 жыл бұрын
What are you struggling with?
@kulpranav2 жыл бұрын
Its just another kv data store. Why is it hard for crud?
@AbHIShEkUPAdhYaYshekup4 жыл бұрын
Perfect.
@galeop Жыл бұрын
13:57
@arifmalikoracledba97573 жыл бұрын
Might it worth blowing your nose (away from the Mic ! ) & clearing you air waves before start of your session !