Amazon EMR Deep Dive and Best Practices - AWS Online Tech Talks

Amazon EMR is the industry-leading cloud big data platform for processing vast amounts of data using open source tools such as Apache Spark, Apache Hive, Apache HBase, Apache Flink, Apache Hudi, and Presto. In this tech talk, we introduce you to Amazon EMR design patterns and architectural best practices. We show how EMR can help you run Petabyte-scale analysis at less than half of the cost of traditional on-premises solutions and over 3x faster than standard Apache Spark. You'll learn how to spin up and spin down clusters as needed for short jobs and how to create highly available clusters that automatically scale to meet demand. Discover how Apache Hudi simplifies building data pipelines. You will also learn how to run EMR clusters on AWS Outposts for on-premises or hybrid deployments.
Learning Objectives:
*Learn how to design a big data environment using best practices
*Find out the best way to support Apache Spark, Hive, HBase and other open source applications
*See how to choose the right approach for both short- and long-running jobs
***To learn more about the services featured in this talk, please visit: aws.amazon.com/emr Subscribe to AWS Online Tech Talks On AWS:
www.youtube.com/@AWSOnlineTec...
Follow Amazon Web Services:
Official Website: aws.amazon.com/what-is-aws
Twitch: / aws
Twitter: / awsdevelopers
Facebook: / amazonwebservices
Instagram: / amazonwebservices
☁️ AWS Online Tech Talks cover a wide range of topics and expertise levels through technical deep dives, demos, customer examples, and live Q&A with AWS experts. Builders can choose from bite-sized 15-minute sessions, insightful fireside chats, immersive virtual workshops, interactive office hours, or watch on-demand tech talks at your own pace. Join us to fuel your learning journey with AWS.
#AWS

[@oldoctopus393]

thanks for demo and presentation, it's rare to find such a professionally structured materials!

[@amazonwebservices]

Happy to help you with the best resources! 🤝 ❤️ 😎

[@friends4lives]

Great Video. To the point, clear and very helpful. THANK YOU!

[@thekaders3535]

This is an excellent presentation providing concise depth on a number of critical features

[@mohitpandey5190]

In scaling comparison time scale is different for both of the graphs(12h vs 14 days), I believe the graph will be much smoother in higher time frames.

[@saleembasha1131]

Fantastic demo and the presentation is rich and cool one ..... its more helpful to design productioniged solutions

[@giovani345]

Rich content. Thank you!

[@Larry21924]

This is a marvel. I read a book with similar content, and it was a marvel to behold. "Mastering AWS: A Software Engineers Guide" by Nathan Vale

[@gunjankhandelwal1586]

This is amazing, appreciate it. :)

[@leandromana]

great presentation, covering key features, well explained and showing demos as well.. thanks very much ; )

[@bugzi11a]

great video. Can we get that Step Functions code ?

[@avitabayansarma1011]

Very informative!As EMR File system provides provisioning s3 services. Can I use EMR FS as the storage and on top of it running spark for handling big data? Is it possible to replace hadoop with s3?

[@kkanu3833]

AWESOME

[@sanooosai]

brilliant

[@vigneshvit]

Am quite intrigued about the small file problem.. .say thr is a table and has 10 recs and we the convert them we create a parquet ifile..whr.woukd it cause problems? While say external table creation or while we run any inquiry or during any analytics job performing complex calculation...

[@mwont]

Is the step function code open sourced?

[@justindebo997]

Diddo on the step functions code. Great video though!

[@wasimbagwan6771]

Helpful video. Link to ASL code?

[@zucbsivrtcpegapjzwrf2056]

text

[@manjuchoudhary6318]

A lot of information for sure but not explained well to actually understand it