Kafka Tutorial - Node.js Producer & Consumer

Learn how to create a kafka producer & consumer using node.js.
We will be running apache kafka in a docker container. We'll create a producer & consumer using node.js which will read & write events to the stream. This is useful when building event driven systems.
Check out the code on Github: github.com/kri...
node-rdkafka - www.npmjs.com/...
avsc (for serialization & de-serialization) - www.npmjs.com/...
Don't forget to Subscribe here: / @krisfoster1

[@nishanthanr605]

no annoying music no extra talks. very clear content

[@pankaj_9998]

Thank you so much. Great explaination. Can you do a simple microservices project in nodejs with kafka being used for async communication among the services. That would be great.

[@amirmohammadmirzaeirad3]

Thanks man, great tutorial for those who are just getting started with kafka with nodejs.

[@MegaJagveer]

Incredible video with very clear steps and instructions. Looking forward to more stuff!

[@KrisFoster1]

Thank you @Jagveer Singh!

[@saqibullah7286]

Nice explanation much recommended for beginners

[@NIKOLAZ56]

impeccable way of explaining how kaftka works

[@imsarvesh_]

Finally I understood Kafka.

[@KrisFoster1]

Thank you, glad you enjoyed!

[@CodeCraftsman]

This is awesome man! I was looking for short and crisp explanation of Kafka! You nailed it!

[@Cyberspider76]

Well done… simple and too the point!

[@KrisFoster1]

Thank you!

[@surajjoshi5503]

Great Explanation

[@andreassef3656]

Thank you for this content, it help me so much to understand how kafka works!

[@objectObject212]

Great tutorial, exactly what I was looking for!

[@ayushpgupta5755]

Very clearly explained with bare minimum code.

[@mohammadpartovi1813]

This was fantastic. Great explanation. Thank you very much for sharing your knowledge.

[@TheCouncellor7]

Thanks Kris, what a fantastic clear instructions and clean code, wonderful to watch, thanks again for sharing your time and your knowledge, much appreciated

[@elad3958]

I subscribed to your channel after this awesome tutorial

[@kal_node]

I found it quite explanatory and helpful, thanks!

[@muratasarslan2359]

Thank you Kris, very clear & helpful.

[@raviparashanaboina4644]

Thank you So much it is one of the best tutorial for producer and consumer great work!

[@ryanhill101]

Nice video Kris and super easy to follow.

[@KrisFoster1]

Thank you Ryan, glad you enjoyed!

[@sreeananthakannan]

It was the good video to understand about basic of kafka

[@JosuaLeonard]

Thank you Kris! Really clear and straightforward explanation. 🔥

[@fraewn2617]

Great tutorial, thank you!

[@burtreynolds111]

Awesome video Kris!! So helpful.

[@babanmalhi6168]

Thank you so much. Very very helpful

[@panjijayasutra1573]

Very usefull, thankyou verymuch Kris

[@hegeltron]

Thanks Kris!

[@mahendranath2504]

Thank you so much ❤️👍🏼🎉⭐🙏🤛, subscribed and 👍🏼

[@AnkitKPunjabi]

Crisp and clear

[@eternalharmony0]

Thank you so much.

[@raymondoyinlola7765]

This is really helpful.

[@ntwariegide5993]

Thanks for this great video

[@karmakiller360]

Good Job. Thank U

[@Kreator321RG]

Thanks a lot! You are the best))

[@herbertk9266]

Thanks for sharing

[@KrisFoster1]

Glad you enjoyed!

[@atiqbaqi]

great example

[@manikjain6048]

Nice work

[@KrisFoster1]

Thank you @Manik Jain, glad you enjoyed!

[@Mvrck44]

Thanks a lot for this. Quick hint, at the time of writing this comment, the 'npm i' won't work as is: new minor version of node-rdkafka(2.11) is available, need to be updated manually.

[@Not256O0]

Hey man, how did you overcome this problem?

[@Mvrck44]

@@Not256O0 after couldn't get neither 'npm ci', nor 'npm i' to work, I installed both dependencies listed in package.json manually, like 'npm i '. This installed node_modules and updated both .json files.

[@karusalavinaykumar7399]

Thank you for the Tutorial. npm i node-rdkafka is failing for us. Please make a video on how to install the pre-requisites and install node-rdkafka. Thanks.

[@thong3233]

i can't install rdkafka, please help me

[@afshinrahmati786]

Thank you ,

[@vitorsantana2795]

Thanks a lot!!

[@chathurangabandarapc]

Thanks you very much

[@daesvideo]

Hi, thank you very much. Can you do another exaple with multiple consumers please?. I tryed but just only one consumer can receibe message. thank you.

[@GM-je3zp]

nice tutorial... Kafka container doesnt run for me though and had to get the compose file from an exmaple from cofluetn

[@get_carrot]

like your voice

[@abderrahmane_24m]

Hello, thank for this tutorial! I'm facing a problem using docker-compose, when producer gets triggered, I'm getting LibrdKafkaError: Local: Broker transport failure, any suggestion?

[@_slier]

so from your demo, kafka is like messaging system like rabbitmq?

[@K_H-Channel]

Hi there, my company has Kafka built in Java. I am looking for a way to automate the events and use assertions to verify the events. Events can be in thousands. Is there a way to automate this using JavaScript/ node js/ Jest/Mocha/Chai or anything else that’s JavaScript? Thank you.

[@andriizilnyk4927]

does not wok on node 14.17.5

[@mohamatnasri9619]

i use type: modules still get same error, so how?

[@rules654]

If in windows and facing issues in creating topic use this command --------> docker exec -it [container_Id] /opt/bitnami/kafka/bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic test you can get the kafka container Id by running ---> docker ps replace [container_Id] with the containerId of the kafka container

[@madeby-jugnu]

getting error while installing rdkafka 😢 anyone help please

[@kishorrathva4097]

Which Nodejs version are you using ?

[@lynnegaming7328]

can i make it without docker??

[@shivammishra-sj8jo]

Hi Kris i am not able to install node-rdkafka can you help me out ?

[@daesvideo]

Hi, search some Dockerfile with RUN apt intsall cmake, and other tools, that's the good one.

[@thetrooper6]

great demo! thank you!

[@KrisFoster1]

Glad you enjoyed, thank you!

[@olorundareagnes934]

Great tutorial. Thanks a lot!