Partial Multi Tenancy on Kafka using Strimzi

Colt McNealy, Founder, LittleHorse
Managing a Kafka platform that serves multiple different applications within an organization (what we call "partial multi-tenancy") is a daunting task. How do you create topics? How do you enforce the principle of least privilege? How do you mitigate the noisy neighbor problem? The good news is that Strimzi has kube-native answers to all of these questions. Apache Kafka is the core dependency for the LittleHorse workflow engine. In LittleHorse Cloud, we use Strimzi (of course) to manage the Kafka clusters that back our LittleHorse Clusters. In particular, the "LH Cloud Enterprise" product uses Strimzi CRD's to enable partial multi-tenancy, in which multiple LittleHorse clusters share a single Kafka cluster. We will cover how the KafkaTopic CRD simplifies the provisioning of new LH Clusters, how we use the KafkaUser CRD to enable tenant isolation, and how we use the KafkaRebalance CRD to take advantage of Cruise Control. Lastly, we will discuss some up-and-coming Strimzi features such as KRaft support with KafkaNodePool's and topic replication factor changes with the Unidirectional Topic Operator. These new Strimzi features will allow us to deliver some great new functionality to our users and we are very excited to see these Strimzi features cross the finish line!