AWS Glue Tutorial for Beginners [FULL COURSE in 45 mins]

[@JohnnyChivers]

Hi Folks - The much requested update to this video with the new AWS Console UI for AWS Glue is now available on the channel with a new GitHub repo containing everything you need to follow along. kzbin.info/www/bejne/kKethJSfpLWMr9E.

[@sayakutube]

you were preparing the video when the entire west world was yelling "Happy New Year"! Great commitment and awesome result!

[@TerminatorAyan]

in 41 minutes, I never knew I can gain this much knowledge and at least be ready with AWS glue as we are in a transformation project. It really helped, thank you so much!

[@farhodotaboev3324]

If you had watched with 1.25 playback speed, it would take you 33 minutes

[@danielboza5747]

For me the most amazing thing is that he was working in the morning of January 1st. My respect! 😅

[@adeyemitunji343]

This the most detailed AWS glue video i have ever seen. Keep up the great work Johnny.

[@waeldimassi3355]

Amazing content by far !! Please continue with Glue. Never seen such high quality tutorial :D

[@JohnnyChivers]

Thanks for watching

[@vedprakash9413]

41 minutes was sufficient for me to get hands-on AWS glue. Thanks, Johhny for this awesome tutorial.

[@augustocarrillo5927]

I don't like subscribing to people but lit I wouldn't be able to learn aws without you. Best and unique content

[@SungSam-wz2vk]

Awesome. I've never met people like you that you're so young with solid data science & aws skills.

[@JohnnyChivers]

Thanks Sung!

[@junaidraza6774]

I am not sure, does anyone said this before or not, but i wanted to say, You are a rock start.

[@chamila.fernando.us2fernan663]

You are Awesome. watching in 2024... ETL steps needs minor updating but I was still able to follow ! Keep up the great work !

[@jairo_55]

I am using this in 2023, the interface has changed, but this still working, thank you so much for the video.

[@hankbirkdale2154]

One of the best tutorials I've ever seen - let alone on a topic that is not easy.

[@StephenGillie]

Troubleshooting tip for especially complex environments: If the user has access to ALL S3 locations in the table, then Glue will assemble the table for Athena to query. If even one S3 location can't be accessed by the user, then the table won't show up in Athena. Hope this helps complex schema lovers.

[@JohnnyChivers]

Good tip.

[@adalke2]

As someone transitioning from a data scientist to DE role, I found this extraordinarily helpful! Subscribed! Thank you!

[@arunsar7893]

I am interested in understanding why did you decide to make the transition.

[@JohnnyChivers]

Awesome! Thank you!

[@ThanhTran-wf2jf]

This is one of the best tutorials i've seen in youtube videos. Very well-explained, very useful. Thank you for uploading this!

[@JohnnyChivers]

Thanks for the comment and supporting the channel.

[@Ajay-nc9yn]

I have spent many hours on Udemy for Glue Job and Glue data Catalog, but after watching your video. I must say Damn Good Stuff, Sir!

[@JulioSerratos]

This tutorial is pure gold. Thanks

[@victors9585]

Johnny, you ROCK, man!!! 🔥👏🔥👏🔥👏🔥 Thank you so much and please don't stop spreading the wisdom of the GURU!!!! 🙏🙏🙏👍👍👏👏🚀🌟

[@parisnakitakejser]

As a skilled Ops Engineer / Developer i have search for infomartion about whats Glue and how its working in the basic and thanks so much for your video, its late me understanding the basic about what Glue are and whats it can be used for, :)

[@guilhermejf8642]

I just can't believe it's free here on KZbin! Great content, very well explained. Thanks!

[@JohnnyChivers]

Thanks for watching the video and a supporting the channel!

[@lyndsaymizen1881]

Great intro to AWS Glue for someone who's never seen it before. Needed to know these basics as we're moving to Glue for our data transformation/warehousing and this video does just that! Thanks!

[@JohnnyChivers]

Thanks for watching Lyndsay!

[@manasisingh294]

you speak my favorite accent that's such a plus for me. Thank you so much for the quality content! :)

[@femaledeer]

Glad the tutorial took me through the console instead of running code.

[@kakamoora7874]

simple and explain everything ... what we need .... thank you man ... i'm from sri lanka

[@EEdgerocks]

excellent Glue tutorial .. One of the best I have come across which teaches in a composed pace and easy to pickup for anybody

[@JohnnyChivers]

Thanks for watching!!!

[@GauravSakhuja]

Great tutorial in 40 mins you covered everything essential in Glue 👏

[@1209youandme]

Superb course, covering all features with examples

[@JohnnyChivers]

Thanks for comment.

[@engenheironomade]

Excelent tutorial. I made three others courses before that and only with this that I understead complete this tool

[@CaseWalker-j4g]

I am enjoying the video, thanks for this resource! I wanted to note that it seems you blurred your S3 buckets early in the video, but at 19:22 and also 21:33 when configuring glue, you do not blur your other S3 buckets. Also 24:49.

[@AbhirupSenguptaUK]

your commitment shows in the quality!

[@jimaustin3608]

At 28:00 (AWS Glue Jobs section) current screens are much different than video. Figure all the same information has to be entered, but order and screen flips are totally different.

[@martinvuong6652]

Such an amazing tutorial! Thanks for getting this together. Looking forward to more content.

[@Josh-og9eo]

So excited to find this channel. Looking forward to watching all your videos!

[@asifalimir3043]

Such an amazing video. Gives end to end idea about Glue in just 45 mins. Keep it up.

[@shelleycurrie764]

Johnny Chivers you rock. Really great intro to Glue.

[@mohamedyasser5285]

Fantastic video, going to binge-watch your videos for the next few months!

[@JohnnyChivers]

Thanks for watching.

[@Pavan-kn5pg]

One of THE BEST videos on AWS Glue. Thank you Johnny :)

[@akashdasgupta5088]

This is one of the best AWS Glue tutorial that I've come across on KZbin. Totally worth a like and sub❤️

[@JohnnyChivers]

Thanks for the sub!

[@diegogiardini]

It was very usefull for someone who is starting with AWS from scratch! Thank you :)

[@venkatrao7868]

You are amazing and a natural teacher !!

[@dbeckett]

Finally an aws tutorial that uses "wee" and talks in a NI accent, made me laugh that I finally found a fellow NI person :) followed every word :D

[@ToddCunningham]

right when i needed this, always amazing content

[@JohnnyChivers]

Thanks for watching Todd

[@Rk23able]

Wonderful and neatly described in such a short video

[@gabesusman4592]

this channels a great resource for learning data engineering on aws, it's been a big help. Keep up the good work Johnny!

[@AviinashP]

excellent video on glue

[@jeevangangavarapu8683]

Excellent work Johnny. Keep continuing the work .

[@JohnnyChivers]

Thanks, will do!

[@ryoutausami6817]

Thank you. This is an awesome introduction to AWS Glue for beginners.

[@shadabbigdel5017]

Thank you very much for you great explanation and hands-on! It was very useful for me.

[@hemantmattoo]

Very nice and easy to learn video especially for begineers. Thanks for posting this.

[@roarmarketingconceptsllc5074]

This was a great tutorial! I really learned a lot about AWS Glue and I plan to leverage this knowledge to be more effective in my job tasks. Thanks so much, Johnny!!

[@debarshiacharya3729]

This was a really nice video. Lot of learning within a very short time. Thanks

[@sriravi815]

Thanks for nice video, it helped me understand glue and setting up the pipeline in aws.

[@davidlean8674]

Great & comprehensive coverage of AWS Glue. I was hoping it did much more. Makes you appreciate how much easier, faster & cheaper it is to develop in Azure DataFactory. Where a single pipeline can consume multiple different CSV files, parse the filename, figure out the metadata on load, & redirect the output based on lookup parameters. All graphical or generated via script.

[@DataEngUncomplicated]

Hi David, This video did not cover AWS Glue Studio which provides more of a compressive graphical interface that I think would be more comparable to Azure Data Factory.

[@bitashamsss]

fantastic video, very helpful and useful. Looking forward to seeing more. keep up the great work and thank you so much.

[@vjsnapp8178]

Amazing Content and style of teaching, Thanks a lot for making these videos.

[@javiermadriz7834]

Great video for beginners I hope to build some projects for keep learning

[@ammar9700]

Nicely done and explained. I have done my first AWS Glue job, without any issue... thumbs up bro 👍

[@JohnnyChivers]

Fantastic news Ammar!!

[@ambhat3953]

Very helpful for my studies

[@obedrajugantala3489]

Good tutorial. I was able to follow and execute. Thanks Johnny!

[@patriciacafundo1626]

Very good course, gave me a good view of glue AWS service

[@vrlchebolu]

Great intro on AWS Glue! Thanks!!

[@sreddy5845]

Fantastic video: I have two questions 1. Can you create a partition key when importing data with a crawler? 2. The UI seems to have changed. The 'ETL job' has changed. Can you publish a refresher on that part?

[@gonzalea35]

This was awesome explanation of glue, thanks.

[@ibinabobob-manuel323]

Towards the end of the course, I lost my way after we created the parquet file because my version is different from yours and I couldn't just get the visuals on my dashboard. Thank you so much and I will give this a try again some other time.

[@hazemzamalkawy14]

Thank you very much, I learnt a lot from this tutorial.

[@KoEDeath]

Impressive free content. Thank you!

[@JohnnyChivers]

Thanks for watching

[@machinimaaquinix3178]

Thank you and well done. Was able to do the course still in June of 2023, though the ETL chapter was a bit of a challenge as AWS has completely redone those screens.

[@JohnnyChivers]

Great to hear!

[@michealdmouse]

How did you get past the ETL job chapter? The interface is totally different.

[@guilhermmontealto2160]

huge fan from brasil, thank you for your great content, it helped me a loooot

[@JasonZhang-se2jo]

You are definitely a hero

[@JohnnyChivers]

Thanks for watching

[@sagnikmukherjee5108]

Thoroughly enjoyed, Thanks.

[@Matrix_Mayhem]

Thankyou so much. Very informative Johnny!

[@maryo1134]

Amazing teacher..Many thanks for this

[@JohnnyChivers]

Thanks for watching.

[@ishabisht3685]

Thank you very much, it was very helpful video to learn end to end AWS Glue. 🙂

[@PURAMSAIPRIYA]

Thank you for this video, it is very helpful gives a clear glimpse of AWS glue. This is a very important video for me for the interviews :)

[@maxmax2746]

Thank you for your work! Could you please help few little questions: 1) i saw, that while you it said that "customer_csv" table is partitioned, but "customer_parquet" is not (there is not "partitioned" mark nearby). So how should i make that files partitioned? Because it seemed, that you did the same thing for csv ant parquet, but get different result. Thank you in advance! 2) What would be happened, when you add another csv data for different day? how job will work? i didn't get, how jobs determine, that previous day already transformed to parquet, and newest day - no. And what if . As i understand - in your particular example from that video, that Glue job will be transforming from csv to parquet all the fyles inside customer_csv folder. But how to make it more determined based on run date? for example: in general i want to transform only previous loaddate files. As i understand, it should be done only in code of job

[@JeffLentz]

Fantastic! Thank you for putting this together. It helped me a lot.

[@sarfarazsiddiqui9683]

csv file is loaded from S3 but after creation the crawler, csv file data is not shown into the ATHENA. It shown only csv file headers not shown the whole data.

[@augustogoldner4267]

That was fantastic, mate! Thank you very much for sharing your knowledge!

[@nishaddheeraj2]

really amazing and fun learning with Johnny :)

[@PatrickMcDonoughVanWash]

Highly recommend!

[@michealdmouse]

Could not follow past the 28:00 mark. If anyone has any advice on how to set up the AWS glue please let everyone know.

[@balledachandrahas8326]

Super helpful.... Thank you so much.

[@JohnnyChivers]

Thanks for watching!

[@robdood2]

Thanks for this! So much better explained than AWS's own tutorials!

[@kjewelson]

happy new year

[@muadddib4734]

Excellent content! Hope you keep it coming. I just subbed

[@JohnnyChivers]

Thanks for watching.

[@Straight-Data-Science]

Very well done! Thanks for sharing this!!

[@danielmdubois]

Do you have any resources or can direct me to any resources that can dig deeper into the topic skipped over @8:50, i.e., what someone would actually do with regards to IAM roles best practices?

[@claudioruz]

Good training course, thank for all

[@JohnnyChivers]

Thanks for watching.

[@qadiralidanish7529]

Awesome tutorial.

[@benitinmagnate4937]

@25:00 "I'll talk about connections quickly", LOL! That's what AWS Glue, Azure Data Factory, SSIS, Informatica, are all about: CONNECTIONS! You are moving data from a source to a target, and to do that, you need to be connected to both, the source and the target. Basically, you are an S3 guy, LOL!

[@vamsikrishna5950]

Awesome content!! Appreciate the effort. Thank you

[@maxpayne6625]

I feel like now I am zero to a noob. It will take sometime to be a hero :)

[@TheSimpGatsby]

thanks mate, u made my day.

[@mgd.kn111]

Love this tutorial, thanks!

[@JohnnyChivers]

Thanks for watching!

[@tullez01]

Amigo, muito obrigado pelo vídeo... Ficou ótimo! Abraços! Dear friend, thanks for this video. It's really great, helped a lot... Hugs from Brazil :)

[@LaurenArmstrong-f7g]

Could you consider updating this course for 2024 and also adding a section for DataBrew? It would be helpful to see you walk through an example of moving a dataset from S3 bucket into DataBrew/Glue and the output being Athena or QuickSight ready

[@jriosfer]

It's very interesting your approach, I have a question, why does the glue's crawler from converts csv file to parquet format could not create the parquets table with a partition definition, which came from csv file?

[@JohnnyChivers]

Hi Jorge. The crawler will only create partitions when there is a folder. So something like s3://table_name/parition_1/file.csv will result in partition being created. However, crawlers can be a bit temperamental. They are some sort of ML algorithm and do go AWOL at times. For that reason I usually just create the tables through code manually - terraform, cloud formation or even DDL via Athena.. when it comes to real life use cases.

[@kicknotes]

Good video. Thanks for this!

[@anarossetto2742]

wonderful tutorial, great explanation, thank you! Unfortunatly aws glue job console has changed a lot and I could not finished the tuto :(