Run Spark Jobs On Amazon Athena [FULL TUTORIAL IN 12MINS]

[@DroisKargva]

Awesome! I just started learning Spark this is ideally on time. Thanks as always Johnny!

[@nishvanth]

Thank you for the video Johnny. Can you make a tutorial on AWS Redshift please?

[@theengineeringsideofdata6246]

Great explanation, Johnny. I remember seeing this announcement at re:Invent and was curious about the details.

[@TheErchetan]

Very Good Content.

[@dhananjaypawar2496]

hello Johnny your videos are best for all data engineer needs. Can you please make detailed videos on lambda and Redshift.

[@dvo66]

Hey Johnny, great vid. I've learned a lot from your videos and they have helped me in my interviews as well. I have a request, could you please make a video explaining different IAM policies used in production for data engineering stack ? I've followed all of your videos but we have used admin access for labs but I want to learn the real IAM policies used for different access in production. Hope you see my comment. Cheers!

[@ToddCunningham]

love your videos, thank you

[@JohnnyChivers]

Thanks for watching Todd.

[@ziauldba]

You are best my friend ❣️

[@shakthimaan007]

Can Athena query Redshift as well? Lets say I need to query S3 by joining a Redshift Table.

[@nitropan]

So all the completed results are stored on the S3 Bucket in results folder? what type of output is it?

[@JohnnyChivers]

It’s csv but there are ways to store it as another format aws.amazon.com/premiumsupport/knowledge-center/athena-query-output-different-format/

[@mohsenimani6652]

Thanks for the tutorial, how can we setup a job to run a Athena notebook on a specific cadence?

[@santiagomorales8806]

Hello Johnny, thanks for the video. What's currently the main difference between a job in Glue and a job in Athena?

[@JohnnyChivers]

Presently there is a large list of limitations with Athena spark… around reads, writes and even what table types it can read. Whilst you get all this functionality with glue. I’ve left the link to the list of current limitations at the bottom of the video description.