Kafka Implementation with Producer and Consumer example in Spring boot | tutorial | Code Decode

In this video of kafka implementation with producer and consumer example by code decode we have explained
Udemy Course of Code Decode on Microservice k8s AWS CICD link:
openinapp.co/u...
Course Description Video :
yt.openinapp.c...
Introduction to kafka : • Kafka tutorial for Beg...
Kafka Architecture : • Apache Kafka Key Termi...
Github link : github.com/cod...
Link to download Kafka : kafka.apache.o...
Commands to start Zookeeper and Kafka :
.\bin\windows\zookeeper-server-start.bat .\config\zookeeper.properties
.\bin\windows\kafka-server-start.bat .\config\server.properties
Kafka implementation with producer and consumer is very important topic with interview point of view.
What is Kafka
Apache Kafka is publish-subscribe based fault tolerant messaging system. It is fast, scalable and distributed by design.
It was initially thought of as a message queue and open-sourced by LinkedIn in 2011. Its community evolved Kafka to provide key capabilities:
Publish and Subscribe to streams of records, like a message queue.
Storage system so messages can be consumed asynchronously. Kafka writes data to a scalable disk structure and replicates for fault-tolerance. Producers can wait for write acknowledgments.
Stream processing with Kafka Streams API, enables complex aggregations or joins of input streams onto an output stream of processed data.
Traditional messaging models are queue and publish-subscribe. In a queue, each record goes to one consumer. In publish-subscribe, the record is received by all consumers.
Pros of Kafka
Loose coupling - Neither service knows about each other regarding data update matters.
Durability - Guarantees that the message will be delivered even if the consumer service is down. Whenever the consumer gets up again, all messages will be there.
Scalability - Since the messages get stored in a bucket, there is no need to wait for responses. We create asynchronous communication between all services.
Flexibility - The sender of a message has no idea who is going to consume it. Meaning you can easily add new consumers (new functionality) with less work.
Cons of Kafka
Semantics - The developer needs to have a deep understanding of the message flow as its strict requirements. Complex fallback approaches may take place.
Message Visibility - You must track all those messages to allow you to debug whenever a problem occurs. Correlation IDs may be an option.
Most Asked Core Java Interview Questions and Answers : • Core Java frequently a...
Advance Java Interview Questions and Answers : • Advance Java Interview...
Java 8 Interview Questions and Answers : • Java 8 Interview Quest...
Hibernate Interview Questions and Answers : • Hibernate Interview Qu...
Spring Boot Interview Questions and Answers : • Advance Java Interview...
Angular Playlist : • Angular Course Introdu...
SQL Playlist : • SQL Interview Question...
GIT : • GIT
Subscriber and Follow Code Decode
Subscriber Code Decode : www.youtube.co...
LinkedIn : / codedecodeyoutube
Instagram : / codedecode25
#kafka #codedecode #kakfainterviewquestions

[@kanishkkumar8272]

Thankyou so so so much for this. I saw this video just before my interview and cracked the interview. Kafka was asked for 20 minutes. Really, thanks a lot. Got a hike of 160 percent . Interviewer was so impressed by the answers. Thank you once again. Really appreciate it. Please never stop these.

[@CodeDecode]

Many congratulations Kanishk. Well deserved Man 👍👍🎊🎊🎊 keep learning keep shining👏👏

[@syedaqib2912]

Congrats man !!

[@prerna3404]

Which company?

[@johnnybreathe]

Thanks for the implementation. I have done synchronous microservices till now. It's totally new concept to learn asynchronous also. 🧑‍💻 Thank you so much. And please continue what ever is important these days and in coming future as a java developer. 🤗 TQ for the videos

[@CodeDecode]

Thanks a lot 🙂🙂 sure we will cover grpc next soon

[@geethufrancis6988]

@@CodeDecode thank uu

[@GauravSharma-up9gs]

Thanks for simplifying everything at one place. One day you must go famous for your videos in your playlist of your channel.

[@CodeDecode]

🙏🙏thanks a lot for these beautiful words Gaurav. These comments means a lot to us. It keeps us motivated to upload more content 🙏🙏🙂

[@ajitpeshane8311]

very simplified and working poc on kafka, Thank you so much. In my case i struggled while executing the command @ 12.06 minutes error: The input line is too long. The syntax of the command is incorrect. Solution: I have renamed the folder to kafka and moved that to D drive directly. worked for me. Might help others.

[@fakruu]

The best video beginner friendly video on kafka + spring boot !

[@CodeDecode]

Thanks

[@kasperamkumar5489]

I didn't went through this video bcoz your videos always rockzz I will go through all your video in a few days. All the best 👍

[@CodeDecode]

Hehe sure 👍👍 keep learning keep shining Man👍👍👍👍

[@RupsiKumari-r8c]

This was very helping in completing my assignments. Looking forward for more basic video. Thanks much.

[@CodeDecode]

Sure we will create more soon

[@SampritiChowdhury-m8m]

Extreme Helpful and helped me to give a valuable KT -session to all my colleagues and make a worthful presentation of it . Thanks a lot mam for boosting Confidence .😇

[@chetankhandave1072]

It's my humble request, please make a playlist for Apache Kafka, so that we can go through each video one by one. It will be very easy for every one. And again you have done a fantastic job. Thank you so much.

[@CodeDecode]

We have one Chetan : kzbin.info/aero/PLyHJZXNdCXset6tBB1aY5aEY77YhbxWLo

[@CodeDecode]

We will add more videos to this playlist 👍

[@chetankhandave1072]

@@CodeDecode Thank you so much for your reply. It will be very useful for every one of us.

[@chetankhandave1072]

@@CodeDecode In this example, consumer and producer are in this same application, but what if Consumer is in one application and producer in another application ? How can we handle it ?

[@ankitgiri6874]

Thanks you so much for all of your videos, The way you explain, every concept becomes very easy to understand.

[@CodeDecode]

Thanks Ankit 😊👍

[@sheiksahil7209]

Great video...loved the way you explained .... Please make a series on Event-Driven Microservices concepts also. Thank you in Advance.

[@CodeDecode]

Sure Sahil. 👍🙂 thanks for the nice suggestion

[@joshinim.n9863]

Thank you so much for this series of videos. I feel so grateful for it. This is so useful. You're really imparting great knowledge in people!

[@CodeDecode]

Thanks joshini for the nice words

[@PranitKothari]

Wow! Awesome. This can't be explained any better, this can't be explained easier.

[@CodeDecode]

Thanks a lot Pranit 🙂🙂

[@nitinvnk3975]

Simple, Crisp and to the point thank you for this content.

[@CodeDecode]

You’re welcome

[@pranjalsingh1099]

Thanks for up to the point explanation. Please continue this series to for explanation of individual components.

[@CodeDecode]

Thanks Pranjal. Sure 👍🙂

[@aman_deep21]

Very nice demo for getting started with Kafka, thank you

[@CodeDecode]

you're welcome aman deep

[@bhaumikpatel5932]

Thanks for explaining apache Kafka. Please upload more videos on kafka streams and interview questions

[@CodeDecode]

sure bhaumik we will create it soon

[@ithisrinu9593]

you are best among all you tuber channels tq madam

[@CodeDecode]

😊❤️🙏🙏

[@sreeramsarath5188]

It's very good and way of telling so .. easy to understand.

[@CodeDecode]

Thanks Sreeram🙂

[@rishisharma3569]

Please please please continue this series. I have subscribed and liked and shared.

[@CodeDecode]

Thanks Rishi. 🙂sure we will continue this series

[@AnuragTripathiShiva]

I have little doubt when you say - “broker is a topic” and “consumer will subscribe to broker”. It conflicts with my understanding that broker is not a topic instead it is a kind of container which have multiple partitions from different topics. And consumer subscribes to topic rather than the broker. Please correct me if I am wrong.

[@lukemagnotta6823]

How Eloquently was the explanation!

[@CodeDecode]

Thanks a lot 👍🙃🙃

[@surpriser2292]

Can you please continue the series and provide more information regarding the Kafka Architecture. Kudos for all the efforts in compiling such great video.

[@PankhuriSinghChauhan-vr9se]

Loved the explanantion.. clear and crisp. Thank you :)

[@CodeDecode]

You're welcome

[@tanveersyed1049]

Thank u so much for uploading one more kafka video

[@CodeDecode]

🙂🙂👍👍

[@AshishYadav-se4db]

Thank you so much for creating Kafka video 😊,and explanation is amazing 🔥.

[@CodeDecode]

Thanks Ashish 🙂👍

[@karthikeyanrm3446]

Please continue this series. If possible make a series on Event driven Microservices. Not sure if you remember - but I have been asking for so long to cover centralized logging, API gateway, Distributed tracing, ELK, docker-kubernetes also (all companies expect this) and then atlast deploy into cloud - AWS/Azure.

[@CodeDecode]

Yeah all in pipeline Karthikeyan 🙂

[@karthikeyanrm3446]

@@CodeDecode thank you so much. Please do. You guys are awesome 😊

[@bpani]

Great session mam, thank you!

[@CodeDecode]

Most welcome!

[@sourabhjinde8191]

Thanks for the detailed video 🙏. Please continue with the architecture video

[@CodeDecode]

Sure thanks Sourabh 🙂👍

[@siddharthpandey835]

Super! continue more videos on Kafka.

[@CodeDecode]

Sure Siddharth 👍👍

[@akashsaha9366]

Thanks maam, for the great explanation

[@CodeDecode]

Thanks Akash 🙂🙂👍👍

[@karamjitsinghsehdev3451]

Nice and Simple. Please try to make real scenario based videos also for experienced. That will help crack interviews. Thankyou.

[@CodeDecode]

Next is too implement kafka in microservices application

[@aishurajas2507]

Thanks alot to you and your fabulous content for java developers! God bless you

[@vengateshm2122]

Very informative hands on tutorial.

[@CodeDecode]

Thanx 🙂👍

[@sagarsonawane3888]

Thanks, very helpful video to understand kafka messaging

[@CodeDecode]

You're welcome Sagar

[@shrutik1863]

Thanks for the video. Please continue series

[@CodeDecode]

Sure Thanks Shruti👍🙂

[@lohir446]

Very useful concept 👌 keep doing

[@CodeDecode]

Thanks a lot ☺ And thanks for the previous comment too 😊

[@dipaliyadav-koli1060]

Thanks for your hard work on this.. As usual best explanation.. ❤️❤️❤️

[@CodeDecode]

Thanks Dipali ❤

[@gudiya1491]

You are a very good teacher i would say 😊, really..very well explained i was struggling with this kafka topic from so long thanks for providing this video.. earlier i watched exception handler ,i understood everything in one go , didn't watch any other videos to understand it better ...can you please provide videos on bamboo and veracode...also aws🙏

[@CodeDecode]

Sure Thanks Gudiya 👍👍

[@arjunyelkachenu]

Great content covered and Im ready to work my Jira story

[@CodeDecode]

Thanks

[@RishavKumar28]

Very well explained!! Please make videos on streams too. Thanks.

[@CodeDecode]

sure we will create it soon

[@saranyavivekanandan9044]

Hi, Thank you so much for your amazing videos. Could you create a video on different microservice listening to this kafka topic, please? I'm quite confused.

[@jayjoshi1366]

It is always worth learning from your videos and this Kafka implementation was really nice one which is even more than MQ. Kindly create next part as well where we can learn about asynchronous communication between multiple micro services. Thanks!

[@rahultripathi1602]

please make Kafka Streams video too and if possible Interview questions video too. Thanks!

[@siddhilalabobo3597]

Excellent video for beginners,

[@CodeDecode]

thanks siddhi

[@user-sapdev]

Thank you so much for for your great efforts and time for getting this content in a simple and clear way. It would be great and useful if you can continue the series..advanced topics like kafka stream API as well.. Thanks once again for your continuous efforts

[@aayanksinghai8242]

Very Well Explained! Thank you

[@CodeDecode]

🙂👍

[@Sedhu_17]

Yes mam we need kafka streams tutorial

[@umamaheswarreddy9204]

i loved your energy , TQ for the video. 😊

[@CodeDecode]

Thanks Uma 🙂👍

[@puneetgahlot3556]

Superb , Wooww amazing.

[@CodeDecode]

Thanks 🙂🙂

[@priyangapinki5715]

Thank you so much. Can you please implement kafka with your vaccination center and citizen project. It will be helpful for understanding real time projects

[@jatinkumarsingh4414]

Great video!!

[@CodeDecode]

Thanks

[@charanm4871]

Thank you so much, this video helps a lot.

[@CodeDecode]

Thanks 😊😊👍

[@cheshvikaraj7769]

nice Explanation. Thank you

[@CodeDecode]

😊👍

[@mukulmangla632]

nice video.. please continue this series. Also can you talk about offset and streams and taking a more complex and practical example

[@CodeDecode]

True. Real beauty of kafka is in streams 👍

[@mukulmangla632]

yes but can also explain offset please

[@unemployedcse3514]

Awesome 😍

[@CodeDecode]

Thanks

[@nikhilpawar7876]

awesome

[@CodeDecode]

Thanks

[@rahulkhot3290]

Thanks for this Nice Video. I could not find the video for kafka streams, could you please let me know the link

[@chetankhandave1072]

In this example, consumer and producer are in this same application, but what if Consumer is in one application and producer in another application ? How can we handle it ?

[@vidhiagarwal9370]

But how to get kafka communicate in different microservices? Would this work? because I see you have done the communication in the same service

[@geethufrancis6988]

Ma'm Thank you soooo much for this playlist. Can you please create a video on setting the kafka messages to Redis db...

[@CodeDecode]

Sure 👍👍

[@kanchankatkar5226]

When will be next video of which you said in last in video, please do keep posting

[@sayanbiswas8847]

Helpful

[@CodeDecode]

Thanks 👍🙂

[@dipeekadeshmukh263]

zookeper and kafka both are not running, giving an error as The input line is too long and The syntax of the command is incorrect.

[@kalyanimvn]

U r rocking mam.asm

[@CodeDecode]

Thanks a lot Kalyani 🙂👍

[@DhruvSharmax]

Can you please let us know how to handle objects instead of string as message. Also how to use kafka streams.

[@abdulrehaman4462]

am facing problem in running zookeeper server , input is too long ,syntax error .... what should i do...

[@ourworldofknowledge5542]

Excellent video

[@CodeDecode]

Thanks

[@bhavyar4983]

Hi mam plz do cover the interview questions thanks a lot for ur time

[@MrGoodLuck]

Mam can you show using docker also? 🤔I tried searching for it but playlist doesn’t have that and also can you please make a video nd explain if I am using okta then how to authenticate only once for all the services 😅or any other method is also fine like how will we apply security for it I have seen the okta video but wanted to know how it will be done in microservices architecture

[@venki1437]

Hai medam, Could please create one video on how to work with jsonb type data in postgress and how to integrate that data with spring boot jpa like how to insert jsonb data in Db using jpa create entity class store data in Db retrieve data from db. Note : there is no videos on KZbin related this concept ,could you Create ASAP. it is helpful to programers

[@sivareddy50]

Nice question, please do a video on jpa concept

[@giridharkumar8176]

I know this is a basic demo, but I have a question, here you did not create topic inside kafka cluster, and no where you added your topic name in any properties file that is being used in two commands u ran in cmd, u added in ur application's properties file but not in kafka application or zookeeper aplication, so then in enterprise application, any one can create as many topic as they want as it does not required kafka broker permission, so how to restrict it then?

[@sureshkamble1526]

Initially title was "KAKA" suddenly wonderd ☺

[@CodeDecode]

🙊our bad. Sorry that was typo 🙊

[@Ammupopproductions]

Please create kafka streams video also

[@CodeDecode]

Sure Anupam 👍👍

[@mash6066]

Hi sister, I have one doubt . In springboot In service class for saving books we will use @Autowired Bookrepository br ; We will use br.save(); Bookrepository is an interface Can u explain ,how it will work . @Autowired we will use for object creation But here Bookrepository is an interface

[@mash6066]

I have already commented , regarding this sister

[@CodeDecode]

This interface must extend some existing interface right? It calls internal api to save your data to db

[@UmmakaJagadish]

madam please do more videos on microservices

[@CodeDecode]

Sure Jagadish 🙂👍

[@atulpatil7472]

Thanks for this wonderful video..I was practicing this code and able to consume my message..after all message consumed. I stop kafka and zookeeper server .once again I have started both server and I am again getting same messages in my consumer. How we can avoid that , as I have already read it before and don't want to read it again as it's duplicate for me

[@resikas.b.5511]

you have to create a topic in the terminal. That is not shown in the video

[@CodeDecode]

Not needed

[@Brainvenus]

Good

[@doctorag1135]

I am getting error group id is not found in exceptation, SO group id can be custom any thing hardcode right?

[@abhishekchaudhari2344]

Can plz you clear why consumer not require deserializer there? when we use deserializer in Consumer?

[@akshayvishwanath2591]

Crazy❤

[@CodeDecode]

😊😊👍👍

[@prateetisingha6729]

Hi , thank you for your videos it's realy helps a lots. Is it possible to share some videos regarding cicd pipeline.

[@CodeDecode]

sure we will upload it soon

[@deepsarandeepu]

Installing kafka and Zookeeper means, Just downloading it from the URL mentioned ?, Some one can explain ?

[@nareshk329]

Please attach PPT as well for information purpose or notes to remember.

[@souvikasus2903]

I am unable to Install Kafka binary. tgz file in my windows 11. Please help me out.

[@abhinavgoel4358]

Can you please explain serialisation and deserialisation in Java with practical example

[@priyanktiwary1067]

Please post video on rabbit mq implementation also

[@CodeDecode]

Sure Priyank 🙂

[@agusp1951]

What about JsonSerializer or CustomSerializer

[@CodeDecode]

We will add them in playlist soon 👍

[@start1learn-n171]

Tq

[@NainaPriya-s9g]

MA'm, i a getting logs in infinite loop in console

[@smehta344]

Hi, I have one question In one interview they asked me What is the Kafka consumer group name? Please can you help me with this question.

[@CodeDecode]

Already gave an example in demo Sanjeev. What I can think of is consumer id here. We can have multiple consumer insame group by giving common group id/ name

[@smehta344]

@@CodeDecode thanks

[@CodeDecode]

🙂👍

[@pradeepgupta2089]

Partition key is missing in sample code

[@rishiraj2548]

🙏👍👍

[@satishsali5571]

Can you give the link of Kafka with Stream video?

[@hamedhamed-ox9ty]

How to get job as Kafka admin can any one tell me only learning Kafka administration will give job?

[@letsthink1490]

Dear please change the title of this video.i really thought KAKA IS a new concept in java😅😅😅😅

[@CodeDecode]

Ohh really thanks for letting us know. 😆it was a typo🙊

[@TummalapalliTejaswini]

I am getting error can anyone help me

[@padmalankipalli593]

Thank you so much @codedecode

[@CodeDecode]

Thanks Padma🙂👍

[@22ankitajaiswal]

Hello codeDecode team, I am getting error when I am doing the implementation by following the video. The error is: Unexpected error in InitProducerIdResponse; The request is timed out. or.apache.kafja.common.KafkaException