Step by Step guide How to Move data from DynamoDB to Aurora Postgres SQL with AWS Glue ETL

[@JeffStoneman]

Thanks for the video Soumil. I have watched several on Glue and yours was very succinct and made sense. Before watching your video, I did not understand that when an AWS Glue Job is connected to an AWS Glue Data Catalog, it is actually connected to the underlying source / target database. That really helped me. I used your video to connect my DynamoDb to an Aurora Mysql db and it worked well. Thanks again.

[@mayurjain2889]

ESSENTIALLY Thanks Soumil for this video. I ESSENTIALLY learn quite a bit from it. ESSENTIALLY all the best for making essentially more videos like this. You are doing ESSENTIALLY a great work

[@SoumilShah]

Essentially thank you 🙏

[@neisha7042]

Thank you for the video, Soumil. It was informative to me and is going to be very helpful for the ETL job that i am trying to write. I had a question, on how to transfer a DynamoDb that using a single-table design to multiple relation db tables? Is it possible to use filter and transfer dynamodb table based on secondary key values to different rds table? I would appreciate some guidance on this.

[@ramachandra7015]

What about complex data exist in dynamo db

[@sahilgupta006]

Soumil, you have shown a POC for a single table, what if the database has multiple tables and we have to transfer the data from RDS SQL Server to RDS PostgreSQL in parallel?

[@OmegaStyler]

What about if I want to load all tables of a postgresql database / data catalog, add the same transformation step to all tables and load them into a new database? How do I do this?

[@johnnydrumgole8476]

Hello im having trouble with creating the connection

[@javeedrahuman4903]

how to do the same this using dynamodb Kinesis stream to aurora Mysql? Can you please make the video

[@azizuddin9269]

if we are pusing data from dynamo to aurora then why do we need to create a crawler for auroraDB? i did not understand the logic here.,

[@catherine4804]

Hi thanks so much for the video, I not sure what I am missing following this, but I keep getting Test connection failed for connection when I try and run the crawler for Postgres. Any guidance?

[@SoumilShah]

Check your VPC

[@ragook3]

Does Aurora and Glue should be in the same VPC?

[@SoumilShah]

Yes they have to be on same vpc

[@ragook3]

@@SoumilShah What is port and ip range I need to open to authorize glue to connect mysql?

[@saikiran4721]

how to handle ddb table with no schema?

[@swathikabalamurugan8689]

Every table has a schema. Schema refers to the column in your table. Hope this answers ur question.

[@SaiKiransk]

DDB tables don't have schema so not every table has schema

[@SoumilShah]

@@swathikabalamurugan8689 Yup that right all table has schema :D

[@iuliansurugiu7762]

Nice and really useful video! You're saying 'essential' too much :)