Spring Boot | Apache Kafka JSON Serialization & Deserialization Example | JavaTechie

  Рет қаралды 25,311

Java Techie

Java Techie

11 ай бұрын

In this tutorial, We will learn How to send and receive a Java Object as a JSON byte[] to and from Apache Kafka using JsonSerializer and JsonDeserializer .
👉 How to configure Serializer and Deserializer using application.yml
👉 How to configure Serializer and Deserializer using java base config approach
#JavaTechie #SpringBoot #Kafka
Spring boot microservice Premium course lunched with 70% off 🚀 🚀
Hurry-up & Register today itself!
COURSE LINK : javatechie5246.ongraphy.com/
PROMO CODE : JAVATECHIE50
GitHub:
github.com/Java-Techie-jt/kaf...
github.com/Java-Techie-jt/kaf...
Blogs:
/ javatechie
Facebook:
/ javatechie
Join this channel to get access to perks:
kzbin.infojoin
guys if you like this video please do subscribe now and press the bell icon to not miss any update from Java Techie
Disclaimer/Policy:
--------------------------------
Note : All uploaded content in this channel is mine and its not copied from any community ,
you are free to use source code from above mentioned GitHub account

Пікірлер: 60
@kynjribajwatmarbaniang8659
@kynjribajwatmarbaniang8659 5 күн бұрын
as coures progress we see less and less views 😀😀😀. but seriously this course is good those of you who completes it helps a lot
@prashantb4220
@prashantb4220 10 ай бұрын
I just finished your entire playlist on Kafka, Amazing content. Thank you for helping me
@fathimashabnamcb6099
@fathimashabnamcb6099 9 ай бұрын
For the first time i completed a tutorial . Thank you so much for this amazing content. Now I got an idea on kafka
@viveksinghbhadauriya7929
@viveksinghbhadauriya7929 12 күн бұрын
Dear Sir, Namaskaram, And first of all Thank You for teaching us so briefly. and also i have been learning a lot from you . the way you have said "IT IS CRYING" while changing String message to Customer customer object at 5.32, i laughed HARD man. Thank You so much sir. its a little thing but i laughed a lot clapping for 5 minutes .. THANK YOU AGAIN SIR
@Javatechie
@Javatechie 12 күн бұрын
Hello Vivek thank you so much for your kind words 😊 and I am so happy to hear that you enjoy the content 😀. Keep learning buddy
@saznajuhirdeen6547
@saznajuhirdeen6547 2 ай бұрын
Kafka series are well structured and theoritical explanation. Superb!!! Keep on doing!!
@ChandlerBing11
@ChandlerBing11 11 ай бұрын
Amazing content!! Thank you so much ❤
@gopisambasivarao5282
@gopisambasivarao5282 10 ай бұрын
Thanks so much Basant, appreciate your efforts!❤
@YoursPravat
@YoursPravat 11 ай бұрын
Amazing❤ Big follower of your content brother😊
@rajenderprasad1193
@rajenderprasad1193 10 ай бұрын
as usual -- awesome.. Thanks Basant
@cloud_Engineer
@cloud_Engineer 7 ай бұрын
Thanks you so much learned apache kafka within few hours
3 ай бұрын
amazing content i followed you since i started my carrier. thanks for you support.
@as_wani
@as_wani 10 ай бұрын
Yeah great content, keep up the good work!!!👍
@techntrade
@techntrade 10 ай бұрын
Amazing series, Please continue this series with Kafka Streams.
@Javatechie
@Javatechie 10 ай бұрын
Yes I will continue from coming weekend
@phanimc11211
@phanimc11211 10 ай бұрын
thanks for ur efforts
@lunatichigh2896
@lunatichigh2896 9 ай бұрын
Thank you
@shubham30372
@shubham30372 6 ай бұрын
Good Content 👍
@rishiraj2548
@rishiraj2548 11 ай бұрын
Great thanks
@pnamm6501
@pnamm6501 10 ай бұрын
another great video! can you cover authentication authorization used in kafka?
@ashvinikakade769
@ashvinikakade769 8 ай бұрын
Hello Sir, You are really doing good job. I am watching your videos and Its really very amazing. Can you please make video on Kafka interview questions?
@Javatechie
@Javatechie 8 ай бұрын
Yes buddy i will
@mysterious5729
@mysterious5729 7 ай бұрын
So it's Crying... got me 😂😂
@gopishettymahindra2713
@gopishettymahindra2713 10 ай бұрын
Thank you so much sir. Can you please upload spring boot annotation second part.
@Javatechie
@Javatechie 10 ай бұрын
Yes buddy i will but required a few more times to prepare for the PPT
@user-pu6ll6xd2b
@user-pu6ll6xd2b 11 ай бұрын
Great please continue. how to get 1000000000 record get from DB and save to another DB by Kafka
@vulo3180
@vulo3180 11 ай бұрын
Great job, on to the next topic. Can you use Kafka Streams or KSQLDB? Thanks for all.
@Javatechie
@Javatechie 11 ай бұрын
I will definitely do that.
@user-ej3re1xs3e
@user-ej3re1xs3e 3 ай бұрын
can you please explain why you create producer factory and kafka template bean? at 19:30 and how that both are working in configuration? I was able to understand kafka template and kafka listener when you create bean in service class in previous section of kafka series.
@ankitratnani3582
@ankitratnani3582 9 ай бұрын
can you please make a video on how to use common pojo and dto classs instead creating same Customer class in producer and consumer projects as you mentioned at 5:33
@sushantsuman4329
@sushantsuman4329 2 ай бұрын
I have one doubt only suppose i m using 10 consumer in consumer_group in kafka all consumer need to implement producer data type class(DTO) to deserializing. any better way to do that because code is repeating in each service . what i mean by that suppose i m producing order data in one of topic in order creation then it is necessary in each consumer class like invoice email and inventory to create DTO class for order to deserializing. Any better way GraphQL? can we use
@NikhilJain08
@NikhilJain08 10 ай бұрын
Hello sir!! What approach can we use for consumers not to connect to kafka directly instead of some api?? Also if the producer wants to move to a new message broker, in that case how the consumer should not be affected by the change at the producer???
@Javatechie
@Javatechie 10 ай бұрын
No if there is a configuration change in the producer then consumers need to have those changes . We can use some centralized configuration approach to manage it to avoid manual efforts
@xyZzzdf
@xyZzzdf 2 ай бұрын
sir have you changed this playlist ? before it was 21 videos, but now only 13
@Javatechie
@Javatechie 2 ай бұрын
No it's 13 only . I haven't uploaded 21 videos. Please check properly
@debabratasarma5945
@debabratasarma5945 4 ай бұрын
At the consumer side, if i am using a config file and not the yml file, with the same configuration and classes like in the yml i.e. the server, string deserializer and json deserializer i am getting an error by the org.springframework.messaging.convertor.MessageConversionException saying : cannot convert from java.lang.String to com.example.dto.Customer. This would occur when the serialization and de serialization do not match or may be something else. What bugs me is if i use the yml file everything runs smooth but when i switch to a config file the consumer application cannot convert the message to Object type. Is there any solution?
@Javatechie
@Javatechie 4 ай бұрын
Please share your GitHub link
@jay-rathod-01
@jay-rathod-01 10 ай бұрын
Bro why didn't we use kafka temple in choreography
@Javatechie
@Javatechie 10 ай бұрын
I have used spring web flux so the sink is used as a consumer over there . If you start implementing the same with the traditional approach then you can go with the Kafka template
@jay-rathod-01
@jay-rathod-01 10 ай бұрын
Got it thanks👍
@vadimsokolov6058
@vadimsokolov6058 Ай бұрын
I have com.producer.dto.Customer for Producer application. And com.consumer.dto.Customer for Consumer application. Since these paths do not match the process does not work. And it is not possible to specify in .yml configuration that the incoming com.producer.dto.Customer should be mapped to com.consumer.dto.Customer. So I have to rewrite one of the applications completely to match the package structure or create a custom Json deserializer.
@machinegunkohli
@machinegunkohli 12 күн бұрын
could you please send rewrited code here
@prajularao
@prajularao 13 күн бұрын
Error:Can't serialize data [com.spring.kafkaproducer.example.dto.Customer@4ad91b0e] for topic [demoCust] I am getting ths error Please help... Below is the config spring: kafka: producer: bootstrap-servers: localhost:9092 key-serializer: org.apache.kafka.common.serialization.StringSerializer value-serializer: org.springframework.kafka.support.serializer.JsonSerializer
@Javatechie
@Javatechie 13 күн бұрын
Please configure trusted package
@prajulakottai1338
@prajulakottai1338 13 күн бұрын
But producer is throwing the error, though I added the trusted package in consumer yml
@Javatechie
@Javatechie 12 күн бұрын
@@prajulakottai1338 you need to add in both producer and consumer buddy
@prajularao
@prajularao 11 күн бұрын
@@Javatechie still not working>please help org.springframework.context.ApplicationContextException: Failed to start bean 'org.springframework.kafka.config.internalKafkaListenerEndpointRegistry' Caused by: java.lang.IllegalStateException: java.lang.ClassNotFoundException: com.spring.kafkaproducer.example.dto.Customer below is the config producerProps.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092"); producerProps.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class); producerProps.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG,JsonSerializer.class); producerProps.put(JsonDeserializer.TRUSTED_PACKAGES,"com.spring.kafkaproducer.example.dto"); consumerProps.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092"); consumerProps.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class); consumerProps.put(ConsumerConfig.GROUP_ID_CONFIG,"group4"); consumerProps.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG,JsonDeserializer.class); consumerProps.put(JsonDeserializer.TRUSTED_PACKAGES,"com.spring.kafkaproducer.example.dto"); consumerProps.put(JsonDeserializer.USE_TYPE_INFO_HEADERS,"false"); consumerProps.put(JsonDeserializer.VALUE_DEFAULT_TYPE ,"com.spring.kafkaproducer.example.dto.Customer"); consumerProps.put(JsonDeserializer.TYPE_MAPPINGS, "customer:com.spring.kafkaproducer.example.dto.Customer");
@kishore87jetty
@kishore87jetty 5 ай бұрын
Sorry to bother you too much, I am using the same deserializer , and I am getting exception its not a trusted source even after configuring. properties: spring: json: trusted: packages: com.example.kafka.consumer.dto It keeps on giving me error on this one Caused by: org.apache.kafka.common.errors.RecordDeserializationException: Error deserializing key/value for partition TestTopic-1 at offset 0. If needed, please seek past the record to continue consumption. Caused by: org.apache.kafka.common.errors.RecordDeserializationException: Error deserializing key/value for partition TestTopic-1 at offset 0. If needed, please seek past the record to continue consumption. at org.apache.kafka.clients.consumer.internals.CompletedFetch.parseRecord(CompletedFetch.java:309) at org.apache.kafka.clients.consumer.internals.CompletedFetch.fetchRecords(CompletedFetch.java:263) at org.apache.kafka.clients.consumer.internals.AbstractFetch.fetchRecords(AbstractFetch.java:340) at org.apache.kafka.clients.consumer.internals.AbstractFetch.collectFetch(AbstractFetch.java:306) Caused by: java.lang.IllegalArgumentException: The class 'com.example.kafka.producer.dto.Customer' is not in the trusted packages: [java.util, java.lang, -java.util-java.lang-com.example.kafka.consumer.dto.*]. If you believe this class is safe to deserialize, please provide its name. If the serialization is only done by a trusted source, you can also enable trust all (*). Posted in instack over flow stackoverflow.com/questions/77804089/unable-to-deserialize-events-from-spring-boot-consumer-getting-an-error-even-aft gitlab.com/kishore87jetty/kafkaproducer gitlab.com/kishore87jetty/kafkaconsumer
@Javatechie
@Javatechie 5 ай бұрын
Could you please share your code in GitHub? I will take a look and figure it out
@kishore87jetty
@kishore87jetty 5 ай бұрын
@@Javatechie Added details in my first comment itself
@kishore87jetty
@kishore87jetty 5 ай бұрын
There are one more branch ProducerConfig and COnsumerConfig in respective repository . the config once are also throwing some error.
@KAMMARIRAJESH151
@KAMMARIRAJESH151 10 ай бұрын
how do we handle this sir? .KafkaMessageListenerContainer : Consumer exception java.lang.IllegalStateException: This error handler cannot process 'SerializationException's directly; please consider configuring an 'ErrorHandlingDeserializer' in the value and/or key deserializer at org.springframework.kafka.listener.DefaultErrorHandler.handleOtherException(DefaultErrorHandler.java:198)
@janyajoshi
@janyajoshi 10 ай бұрын
Can you share your git link. I'll try to see if I can help.
@KAMMARIRAJESH151
@KAMMARIRAJESH151 10 ай бұрын
@@janyajoshi rajeshoo7/Kafka-project/tree/master
@mohamedsedahmad585
@mohamedsedahmad585 10 ай бұрын
did you find the solution ?
@gayannanayakkara8386
@gayannanayakkara8386 10 ай бұрын
I found the solution. its because of type mapping. JavaTechie doesnt get it as producer DTO and consumer DTO is in same package. You should provide like this. is my DTO. Producer : (you should mention your DTO package name in producer as type with the Class name) properties: spring: json: type: mapping: customer:com.gnanayakkara..dto.Customer Consumer : (You should mention where is the mapping Class available in Consumer side.) properties: spring: json: trusted: packages: com.gnanayakkara.kafkaproducer.dto type: customer:com.gnanayakkara..dto.Customer
@Javatechie
@Javatechie 10 ай бұрын
Awesome and appreciate your findings, but just keep a note this is not a recommended way to send raw objects as a Kafka message. I will upload a video on it . Hints : Kafka provides us with flexy that we can deal with record with any data type
Watermelon Cat?! 🙀 #cat #cute #kitten
00:56
Stocat
Рет қаралды 47 МЛН
DELETE TOXICITY = 5 LEGENDARY STARR DROPS!
02:20
Brawl Stars
Рет қаралды 22 МЛН
Apache KAFKA Tutorial | KAFKA Crash Course | Spring Boot
56:49
Daily Code Buffer
Рет қаралды 66 М.
Apache Kafka® Producer Example using SpringBoot 3.x | Java Techie
25:59
Using JSON IN C#! Serialization & Deserialization made easy!
14:47
tutorialsEU - C#
Рет қаралды 25 М.
Serialize and Deserialize JSON using Jackson
35:51
The Life Code
Рет қаралды 6 М.
Java Spring Boot 14 Years Interview Experience [God Level Skills]
54:49
Ждёшь обновление IOS 18? #ios #ios18 #айоэс #apple #iphone #айфон
0:57
Iphone or nokia
0:15
rishton vines😇
Рет қаралды 1,8 МЛН
Урна с айфонами!
0:30
По ту сторону Гугла
Рет қаралды 7 МЛН
Cadiz smart lock official account unlocks the aesthetics of returning home
0:30
Gizli Apple Watch Özelliği😱
0:14
Safak Novruz
Рет қаралды 2,9 МЛН
Secret Wireless charger 😱 #shorts
0:28
Mr DegrEE
Рет қаралды 546 М.