Amazon EMR - When to use EMR on EC2, EKS, and Serverless

[@mertsevenz]

Hey @dacort, Thanks for the great video. - What about Glue? Can we say that Glue and EMR serverless do more or less the same thing? - Let's say we only have Spark jobs to run based on some triggers. Since it is a transient job, I should run it with EMR serverless. On the other hand, if I need a long-running cluster, I should go with EMR on EC2/EKS. Can I extract the formula like this :)

[@HenryLiang-z4o]

the video talks about the advantages of using EMR on EC2 and EMR serverless, so what is benefit of using EMR on EKS?

[@dacort]

EKS (Kubernetes) is great for want to share your compute/memory resources across different variable workloads. Many orgs are adopting k8s, so EMR on EKS helps make it easier to run EMR workloads (like Spark and Flink) on top of EKS.

[@nathanbenton2051]

indeed@@dacort. but one of the catches being that without quota or limit thresholds set at the k8s level, it's very easy for various team/apps to cripple resources in the "emr" namespace for emr containers. anyways, great vid and thanks for the content!

[@jenjayhsu1671]

why EMR serverless does not support Flink? and also why EMR on EKS does not support Hive?

[@dacort]

Each deployment model of EMR has different use-cases and customer bases. In other words, "folks that tend to run a modern k8s environment, also run modern workloads like Spark or Flink, but not Hive."