KubeFlux: An HPC Scheduler Plugin for Kubernetes - Claudia Misale & Daniel Milroy

Don’t miss out! Join us at our upcoming hybrid event: KubeCon + CloudNativeCon North America 2022 from October 24-28 in Detroit (and online!). Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
KubeFlux: An HPC Scheduler Plugin for Kubernetes - Claudia Misale, IBM T.J. Watson Research Center & Daniel Milroy, Lawrence Livermore National Laboratory
Adoption of cloud technologies by high performance computing (HPC) is accelerating, and HPC users want their applications to perform well everywhere. While container orchestration frameworks provide advantages like resiliency, elasticity, and declarative management, they are not designed to enable application performance to the same degree as HPC workload managers and schedulers. In response to increased interest in scheduling flexibility, the Kubernetes community developed the Scheduling Framework to facilitate integration of new policies and schedulers. We present KubeFlux, a Scheduling Framework plugin based on the Fluxion open-source HPC scheduler developed at the Lawrence Livermore National Laboratory. We discuss uses for KubeFlux and compare the performance of an application scheduled by the Kubernetes default scheduler and KubeFlux. KubeFlux is an example of the rich capability that can be added to Kubernetes and paves the way to democratization of the cloud for HPC workloads.