How to Query Your DynamoDB Table with SQL using Athena

[@DisasterFixion]

Thanks man, great video, good catch with the upper/lowercase field names!

[@BeABetterDev]

Thanks and glad you enjoyed!

[@mariumbegum7325]

Fantastic explanation!

[@BeABetterDev]

Thanks Marium, Cheers!

[@marian6040]

Great explanation. Really appreciate it. Thanks

[@christianibiri]

Great video Daniel! thank you

[@AmruthGowda7]

Great Video It would be great if you can do the same video for Large dataset

[@BuildOnAWS]

Great video!

[@SergeyGrebenkin]

Very helpful, thanks!

[@Amanullah-lt6fq]

very helpful. Thank you

[@fehmianac]

Great video Thank you for your effort. I have a question, When we use single table design in DynamoDB, How we can separate partitionso glue table.?

[@CeciliaMoguelMendez]

very helpful!!

[@mmurray4043]

Great video, I think I would have a source bucket with the json and normalise the data using glue into another bucket that I would point Athena to. This way you can just use standard SQL

[@flo_bady]

Hello Daniel, great content as usual. What would you recommend to have it real time? Glue crawler is really asynchrone from what I know. Also to flatten Dynamodb schemas have you already used dynamic frame + relationalize? It does an amazing job

[@BeABetterDev]

Hi @flobady Thanks for commenting. For real time, I would suggest using DynamoDB Streams + Kinesis Firehose. Streams to capture change events or "NEW_IMAGES" of your records, and Kinesis firehose to buffer your data and deliver to S3 in chunks. Then, you can set up a glue crawler on a recurring 1min timer to periodically update the new data. Note that there is a new featue Glue that allows it to detect only the records that changed in your bucket and not the entire bucket. This makes it much more cost effective for incremental additions. Also, never tried dynamic frames yet, but thanks for sharing - I'll have to look into it. Cheers and hope this helps, Daniel

[@flo_bady]

@@BeABetterDev thks for your feedback

[@dipuroy5613]

My DDB table was empty. Then I exported DDB to S3. Inserted some records in DDB. But not getting those new records into S3. Shouldn't it automatically sync the updated data to s3? If not, what to do to get the latest data into s3

[@anubhavsrivastava9909]

HIVE_INVALID_METADATA: Glue table 'x.data' column 'item' has invalid data type:

[@sufga2803]

Thank you

[@sarahwbs]

I don't understand why you would do this. Why have a NoSQL database if you want to use SQL?

[@BeABetterDev]

Hi Sarah, this is a good question. Often times when working with NoSQL, we may still want to perform some analytics or auditing of our data. This type of setup allows you the best of both worlds - NoSQL for great key value performance, and also an ad-hoc data auditing ability with Athena. Hope this helped clarify, Daniel

[@akhan3682]

Excellent content but holy crap that white background [in every video] is brutal on the ole eyes for people with larger 4K gaming monitors. I can't do more than 20mins of yr content a day due to that.