AWS Tutorials - Introduction to AWS Glue DataBrew

[@4niceguy]

Ye... I really appreciate all your wonderful classes.

[@enidaguja6655]

Thank you for all tutorials. They are great and helpful! I am wondering could one project have many recipes? Best regards

[@AWSTutorialsOnline]

It is one recipe per project. However - one recipe can be used in many projects.

[@bobbrandt3043]

Is it possible to merge the columns WITHOUT deleting the original columns?

[@AWSTutorialsOnline]

yes. You can create a new column with concatenation.

[@charliehunter6387]

Awesome tutorial! when i run my Databrew job i get multiple csv's called 'XXXX_part00001', 'XXXX_part00002', etc. Is there a way i can make it output one csv with all the parts?

[@AWSTutorialsOnline]

Out of box, the files are partitioned based on size and you cannot control it. Idea is to not to make too big or too small file as both degrade performance if you try to access data using Athena. You can also configure column based partition. Why you want to avoid partitioning?

[@sridharvuligonda319]

Can we use this DataBrew as an ETL tool instead of Glue studio?

[@AWSTutorialsOnline]

You can. You can convert the recipe into Job. But keep in mind, you don't have any control over code generated by DataBrew and you can change it.

[@sancho709]

Very, very good

[@AWSTutorialsOnline]

Thank you very much

[@krishnaprasadas8566]

What is the underlying processing engine for DataBrew ? I mean where is the jobs, data quality jobs, running.

[@AWSTutorialsOnline]

It uses managed server for this purpose but I am not sure about processing engine configuration. It could be Apache Spark. Sometime - I get errors with DAG mentioned which gives me idea that it might be using Apache Airflow. but I am not 100% sure.

[@krishnaprasadas8566]

@@AWSTutorialsOnline Okey, But don't know whether they really support BigData cost effectively.

[@katiushkaflores]

They even used the same vocabulary as Trifacta!

[@AWSTutorialsOnline]

sorry did not understand your feedback.