Kafka Topics and Partitions

Learn about Topics, particular streams of data, and Partitions, parts of the Topics!
If you want to learn more: links.datacumulus.com/apache-kafka-coupon
Get the Learn Apache Kafka for Beginners v3 course at a special price!
Don’t forget to subscribe to get more content about Apache Kafka and AWS!
I'm Stephane Maarek, a consultant and software developer, and I have a particular interest in everything related to Big Data, Cloud and API. I sat on the 2019 Program Committee organizing the Kafka Summit. I'm also an AWS Certified Solutions Architect, Developer, SysOps Administrator, and DevOps Engineer.
My other courses are available here: courses.datacumulus.com/
Follow me on social media:
LinkedIn - www.linkedin.com/in/stephanemaarek/
Twitter - stephanemaarek/
Medium - medium.com/@stephane.maarek

[@StephaneMaarek]

If you want to learn more, check out my Apache Kafka Series - Learn Apache Kafka for Beginners v2 course : links.datacumulus.com/apache-kafka-coupon

[@rajeshchoudhary1866]

This topic was very clearly partitioned. :)

[@AndroidFish]

Any chance you can do a video on topic design? How would you create topics + partitions if you had a million iot devices all reporting the same thing. Would it be 1 big topic with a load of partitions? What are the best practices?

[@SpiritOfIndiaaa]

thanks a lot , really excellent , can you make vid on how to design/decide number of partitions of each topic ? and also on Kafka connect and confluent?

[@onlysingh7587]

very briefly explained, thanks

[@garfield16591]

Nicely explained. Thanks!

[@kennethcarvalho3684]

Excellent presentation and explanation...understood it perfectly..Thanks

[@hedyehgeranmayeh9194]

Hello, thank you for your excellent videos and explanation. Is there any chances that we can get a message's partiotion and offset due to store in a datasource?

[@ravikirandesai8116]

Hi Stephane, is it possible for you to add a number as prefix, something like Kafka-01, Kafka-02.... so that anyone newly learning Kafka can learn it in order.

[@MrKiraBR]

Awesome explanation, thanks!!!

[@MsZokari]

great explanation 👍. Thank you.

[@venkataramakotibandaru2169]

Could you please give me solution for the below scenario - Ex. In Amazon (Producer), lot of customers will place the orders (Consumers), when amazon wants to send an order update (specific to the ordered person only), how to do this in Kafka - Producer need to filter the consumer and send the message to that particular consumer.

[@MohammadFaizanKhanJ]

Why multiple partition used in Kafka?

[@Oleks380]

Very well explained. Thanks!

[@niravraval3148]

"Not authorized to access topics: [Topic authorization failed.] ” using ssl anyone get any idea about this error. i am using nodejs with kafka

[@vivekpjadhav]

defult retention is 7 days

[@dimness6913]

Can you share ppt?

[@Chand-rn5ek]

nice explain , thanks you ..

[@Indian1947-o1z]

Thnakyou Bro

[@sadeeli]

In this video you mentioned that default data retention for data in a partition is 2 weeks. In another video of yours kzbin.info/www/bejne/laKUep2Bas-dr7c you mention that it is 1 week. I also found out that it is 1 week.

[@josephmbimbi]

bon cours mais c'est "l'anglais forcé" c'est un peu dommage.

[@leoni7649]

I think it sounds great.. better accent than the most french people and waaay better then, idk, Indian accent (which I have nothing against since sharing knowledge is always a good thing)