Best AWS Services You Need To Know As A Data Engineer - How To Become A Data Engineer

[@tomastruchly9484]

I would say AWS Glue belongs to the DE stack. Then there are more specific services like AWS Secrets Manager for storing sensitive credentials from Python scripts.

[@SeattleDataGuy]

Yeah I don’t know why I skipped glue! Pinning comment because I do think glue should have been included!

[@gikis1]

Hey there, thanks for the video! I believe you forgot to talk about MWAA, but your video helped me to reduce the anxiety about learning AWS.

[@SeattleDataGuy]

Yes MWAA is also a a great solution for airflow, but it can get expensive, depending on the size of your company

[@plartoo]

Thanks for the video and really looking forward to the series where you demo how to build ETL pipelines using these tools.

[@SeattleDataGuy]

Yeah! thank you did you see the most recent video - kzbin.info/www/bejne/d4nTgKGmg8xmmbM

[@SeattleDataGuy]

If you guys want to learn more about data engineering, then sign up for my newsletter here seattledataguy.substack.com/ or join the discord here discord.gg/2yRJq7Eg3k

[@supermarkethobo9567]

What confuses me is all the different services that can do the same thing. Like for data orchestration I think of Airflow/MWAA, but then here you suggested Lambda+EventBridge, then I've also heard of AWS Glue which can probably do the same thing.

[@SeattleDataGuy]

This is one of the many challenges of working in tech/DE...So many things that do the same-ish thing. So the question becomes, what do you want out of a solution. Do you need a simple script automated, do you have a complex flow of tasks, do you like some form of drag-drop low code solution?

[@supermarkethobo9567]

@@SeattleDataGuy I just hope recruiters don't throw my resume in the trash because I didn't use their exact tech stack

[@zxcv7586]

do you have an end to end project series where in you're using these aws services for our learning reference? ingestion to s3 using lambda then glue then redshift something along those lines

[@thedailyepochs338]

Heyy i am currently using one of your de roadmaps, would you still suggest a beginner learn sftp on their earlier projects, I am asking based on your opinion of sftp on one of your articles in your newsletter.

[@bobcatmania]

What's the difference between a glue job and data bricks spark?

[@rize3998]

Glue is key

[@xthaone]

Was expecting a Half-Life reference. Nope, it's a AWS service.

[@SeattleDataGuy]

How many people were expecting a half-life reference

[@pedrockc]

I'm pretty good in Half-Life can I interchange that skill with AWS Lambda?

[@SeattleDataGuy]

I was like what? The game, then I googled it. Wild both come up with the same logo

[@walterwhite4365]

😂

[@sriramkumar5531]

🤣🙌

[@Godsfavouritechild_555]

Hi, hope you are having a good day. I have been applying to many companies for Data Engineering roles in the US. But I am not getting a positive response. Can you please help me out by just reviewing my resume once! Thanks a lot for reading out my comment. I know you're busy. Thanks again!

[@anandreddy2992]

your a bit faster, couldnt catchup your pace