Running Kafka in Kubernetes: a practical guide by Katherine Stanley

The rise of Apache Kafka as the de-facto standard for event streaming has coincided with the rise of Kubernetes for cloud-native applications. While Kubernetes is a great choice for any distributed system, that doesn't mean it is easy to deploy and maintain a Kafka cluster running on it.
At IBM we have hands-on experience with running Kafka in Kubernetes and in this session I will share our top tips for a smooth ride. I will show an example deployment of Kafka on Kubernetes and step through the system to explain the common pitfalls and how to avoid them. This will include the Kubernetes objects to use, resource considerations and connecting applications to the cluster.
I will also discuss useful Kafka metrics to include in Kubernetes liveness and readiness probes. Finally I will introduce some of the tools available to help automate the management of your Kafka deployment.

[@aurorajunier5505]

Very nice talk Katherine. It's very informative and well built. Thanks!

[@clipbastler]

Great talk, context well explained & all very clear!

[@asganaway]

that's a great talk, spot on!

[@vikrampancholi2919]

Very well organized and presented.

[@mohiddinshaik3027]

Thank you for detail insight as how to run kafka on k8s

[@wasimhakam4160]

Well explained!

[@mehrdad3141]

Great talk!

[@seenimurugan]

Good one, very neat.

[@mitesh6872]

Can you please provide link to setup kafka on GKE?

[@satishvutti]

In k8s world, how can we make sure that if kafka pod crashes and if some client creates a new channel outside/independent of kafka then before kafka pod comes up how to ensure that new channel is also added or is there any deployment strategy to make sure if kafka has all assets created before new pod spins up

[@sachinupadhyay217]

While trying to cosume the kafka message on same environment unable to do but in local host i am able to do in same environment. In details: when I try to write "hi" txt on producdr its is not sending to comsumer. Getting Null key value ref kind of error. Any solution approach???

[@mikemohd1511]

hey great talk if you able to share sample deployment files that would be helpful

[@manishalankala1622]

any github link

[@vadimprudnikov9072]

Thank you for the talk! Couple of moments were too complicated to understand.

[@KundanKumar-mj8fe]

please give github link

[@EltonMachadoongoogle]

nice talk but I'm a bit nervours with that amount of saliva :/

[@MinhHoang-fu7tt]

Well explained!