Intro to Amazon EMR - Big Data Tutorial using Spark

[@jovelynobias5422]

I hope you create more videos about AWS services. Loved the way you explain things, perfect for beginners.

[@JeffSylvan-y8j]

This is an outstanding tutorial. Thank you for making this!

[@brijeshhota550]

Best tutorial I've seen so far. Was confused between Glue and EMR for a future projects requiring big compute power with control over each node.

[@Munk-tt6tz]

So sad your channel doesn't have more tutorials like this :( thank you so much!

[@MKtechnoverse]

Same thoughts me too

[@DarshilParmar]

Great work mate, very crisp!

[@jayzern]

Thanks man!! Love ur content

[@isaaclee3714]

This is so goood :). Please keep making these kind of videos! Hello from Seattle

[@jayzern]

Thanks Isaac from Seattle! Appreciate your support

[@harishchitluri3137]

Absolutely enjoyed watching the entire video. I felt this video is gonna be great start to understand EMR. Thanks for making it jay

[@miguelhermar]

We need more videos Jaaay 🙏🏻💪🏻 You're awesome dude!

[@meghasingal7082]

Very well explained EMR video, thank you

[@vineethdas4160]

awesome explanation, simple , subtle and to the point!

[@dianadai4616]

I can see you are brilliant.

[@AleLopesYTube]

Excellent tutorial

[@TheDataArchitect]

That was FAST, you are subscribed :D Any vids related to "Amazon Managed Workflows for Apache Airflow"???

[@jeahyunkim3141]

thank you!! I watched the KZbin demo and it was really helpful. I also want to study spark on eks

[@yutao1982]

Very clear! Thank you for sharing this excellent tutorial!

[@szhongy]

great tutorial! can’t wait to see more

[@StartDataLate]

this is crazy ❤❤❤ wish i had seen this earlier ! is this how the whole amazon product in a actual work flow look like？ and also could you maybe make another showing azure system? pleaaase

[@lucashoww]

gnarly stuff man! great content.

[@carloshenriquekaphos8814]

Go ahead bro....CONGRATS TUTO

[@prabhathkota107]

Very well explained, kudos

[@tatenda_mk]

Great tutorials! thanks for the headup! do you have a git repo or more notion notes? would like some guidance

[@vmmismagic]

Hey, thank you so much!!.. you really explain very well!

[@nellyoi9831]

thank you, this is great tutorial

[@thanhchien1602]

Your video is very interesting! Hope you release many new videos :)

[@jasonyuen105]

nice job, great tutorial

[@NhungNguyen-wh7uf]

Could you share more about project for data engineer beginners? I have start to learn to be a DE recently and I hope to know more about some personal project that help me to enhance my skills. Thank you so much for your sharing and waiting for your next video :> Have a good day

[@Ярослав-ю9н8з]

impressive and informative video, good job, go on doing tutorials plss :) Would be very interesting to see a video about spark and snowflake on your channel!

[@hoangng16]

I'm learning data engineering and I find it's hard to find end-to-end data engineering projects to learn building scalable B2B data infrastructure systems that process large amounts of data. Many examples only touch the surface or handle small datasets (well, they say example). Do you have a plan to make a tutorial about actual use cases of data engineering (large and complex data, scalable, cost efficiency system, etc.)?

[@jayzern]

Yea for sure, I’ll consider making more end to end videos in the future. Hard to mimic actual use case data engineering with toy projects, but will try to bridge that gap

[@hoangng16]

@@jayzern, I'm doing a project to provide a management solution for small businesses. Data such as customer appointment schedules, staff's working hours, etc., will be stored in MongoDB. I'm thinking of a data engineering component: some data will be extracted, transformed and loaded to AWS S3 for visualization on a mobile device. I think it's still simple but, perhaps, good enough

[@martinghiena5270]

You killed it. Loved it! Extremely useful

[@jayzern]

Thank you man! Hope to create more

[@goumze]

Great Article ! Thanks for sharing..

[@markkevnjflores]

this is amazing, thank you!

[@mahmoudfadaly8074]

the type of video that makes me wanna quit the field because of how bad i feel about the level I am in , but its a very helpful video though

[@shafiq_ramli]

Are you working as data engineer right now?

[@mahmoudfadaly8074]

@@shafiq_ramli I am looking for a job now as a Data Engineer

[@mahmoudfadaly8074]

@@shafiq_ramliif u could help me find one 😢😅

[@bishop9168]

Fantastic tutorial indeed! I did as instructed and I got two fails in deploying the 'add step' part of the EMR Cluster stage, any insights would be appreciated.

[@123Bankai123]

Hi bishop, I got this error too just now - did you manage to fix it? afaik I did everything the same

[@KheireddineAzzez-l3g]

nice, keep going

[@hassanlaqrabti4036]

More tutorials 🙏

[@errrbrrr3821]

great video! can you make also for AWS Glue? Thank you!

[@BhavyaJoshi-r4z]

Great video

[@pottamvivek]

Great job

[@tatenda_mk]

when writing the spark script, does it ever change or the skeleton layout remains the same? i truly appreciate this and i cannot wait for more

[@jazzypants4047]

I am wondering if I only needed to do PySpark, is EMR the best tool or is it overkill and Glue serverless would be good enough with a lot less to manage and fewer configurations to worry about. Is it possible to enable better performance with all the options in EMR?

[@jazzypants4047]

And thank you for this video - I’m studying for AWS certification and it was helpful to see your demonstration

[@_its_ck]

More videos on Streaming, Airflow and Spark

[@syedmehdi5125]

I hav done masters of science in biotech, 38 yers of age, want to switch to data science...how shud i do it??? Plz reply.....

[@CK30585]

Do projects and add them in your resume. Try upwork and do some projects as freelancers. Keep applying

[@Ved3sten]

Don’t

[@syedmehdi5125]

@@Ved3sten y , plz reply...

[@Ved3sten]

@@syedmehdi5125 bc most companies want senior data analysts or graduate students when it comes to data science. You’ll waste more money chasing a data science job than you’ll make

[@datexland]

Thanks for sharing man 👌

[@shivaramthallapally369]

From where you learn that coding part 😢

[@EstebanHenryG]

Great!! Thank u so much!

[@ZyklonB-88]

why do you need to create a VPC?

[@etf_chach]

VPC is for nodes. It allows them to communicate between each other and the master node.

[@giovannimaia9652]

Please post more videos

[@shakendra2011]

Hey why did you create vpc?

[@jayzern]

Hey, we typically create VPCs over EMR clusters for more networking control and better security. If I rmb correctly, here we defined a public subnet and internet gateway which connects to S3. You could also use private subnets to avoid attaching internet gateway, but it's trickier to setup and can incur NAT gateway charges. The video is just an example

[@mandata143]

is this free to use or do i need to have a licensed software in order to use? this is quite interesting.

[@pradeepnim3689]

Thanks .. Good stuff

[@jovelynobias5422]

Isnt using EMR notebook one of of the ways to trigger EMR job?

[@jayzern]

Yes it is! Wanted to keep things simple in the video so didn't include it

[@tbd4156]

💗

[@atreushouse8848]

bro thank you i survive

[@sisami2109]

thanks for the video

[@moverecursus1337]

Great VIdeo

[@carloshenriquekaphos8814]

Don't stop

[@SaurabhKrPathak]

Just for the information to all the learners this is not how things to be done in tech industries....you need to understand Terra form scripts along with jenkins which deploys aws services....you will not get access to go on management console and play around and do stuff.

[@hoangng16]

Your content is valuable but I find that your presentation is too fast. Sometimes, you fastforward steps too quickly. It makes the video look not lengthy but it's challenging for audience to closely follow your steps.

[@jayzern]

That’s super helpful feedback, I’ll try to slow down on the steps and talk more in future videos. Hopefully my latest video is slower. How the audience feels matters a lot

[@DivakarJ-gk6op]

nice try but its not working

[@jayzern]

Let me know how I can help

[@DivakarJ-gk6op]

I can add a step for the spark application@@jayzern

[@jayzern]

Check if 1. the Spark script is encrypted when you upload it inside S3 2. any typos (line 41 should be "add_argument")

[@DivakarJ-gk6op]

I had tried. but it's not working for me @@jayzern

[@jayzern]

Send me a DM on instagram @jayzern or linkedin, happy to pair up

[@koliux1]

eah good in EMR AWS but an absolute rookie in Videography and equipment use manual focus since you are stationary.... your autofocus keeps looking for something and change light set-up

[@jayzern]

Fair point 👍 will work on lighting and camera setup more next time

[@chulada03]

thanks so much

[@christinachen9669]

Love the ways how you demonstrate! so clear and easy to understand! Thanks for sharing @jayzern