AWS Tutorials - AWS Glue Data Quality - Automated Data Quality Monitoring

[@mranaljadhav8259]

Welcome back sir, waiting for your more videos .. I learned alot from you... Thanks for providing this tutorials for free

[@AWSTutorialsOnline]

So nice of you

[@lucasoliveira7309]

Great video, i was already going to resolve that with a lambda, so more easy with glue data quality, thank you

[@arun.ayilliath]

Great demo! The retry count should have been 0 to prevent re-running.

[@AWSTutorialsOnline]

agree. I realized it later

[@pathakhemant-eb3du]

hey I love your tutorials, Thank You for making our life simpler. so I want know that can we do data warehouse testing with this tool when tables is in Redshift

[@AWSTutorialsOnline]

Currently it support hive metastore with S3 bucket only.

[@hsz7338]

Thank you for taking us through the new feature that AWS Glue offers. Do you see Glue Data Quality replacing Glue Data Brew, at least from the Data Quality perspective?

[@AWSTutorialsOnline]

I don't think data quality in Brew will be replaced. Both will exist. Brew is more for adhoc data preparation and Glue job for automated. Both need data quality feature for their purposes.

[@yinggamonkulsarapitak7948]

Great vid! Thanks! Can this Data Quality integrated with CI/CD and Terraform?

[@AWSTutorialsOnline]

You mean to be able to configure Data Quality using infrastructure as Code. I am not sure - I did not check CloudFormation or Terraform. But it does support APIs for sure.

[@arunr2265]

Welcome back brother. waiting for your videos. Hope everything is fine

[@AWSTutorialsOnline]

All good. Sorry for a long pause from my side,

[@GoofyAIVideos-m9k]

Is it possible to this entire thing using boto3 in python

[@scotter]

In your Glue demo, it *seemed* you skipped showing a part. How did you get from a file being dropped into the S3 bucket/sales to it becoming a table? I'm looking for the most code-light way to set this up so my lambda will somehow be triggered once the file is turned into a table, so my lambda can then run the rules defined in console and then write a log file to other S3 bucket/folder of which rows/columns failed. Thank you!

[@AWSTutorialsOnline]

The table is created using AWS Crawler. I did not mention that because I have covered than my other tutorials.

[@chengchangyu]

thanks for the video. very details.

[@AWSTutorialsOnline]

Glad it was helpful!

[@suttus007]

Hey Sir could you please help me with the certification requirement for this DQDL aws glue , I am looking for the relevant certification for this one

[@cheluveshab9525]

Hi Brother, I’m a big fan of yours. I have learned many things from your channel and thanks a lot. Please provide your LinkedIn.

[@AWSTutorialsOnline]

Many Thanks, sorry for a long pause from my side.