1.2 Billion Records Per Hour High Performance Kafka and Spark - End to End Data Engineering Project

[@CodeWithYu]

Please don't forget to like and subscribe! ❣

[@souniksadhu4266]

Please inform the system requirement to run this project

[@sarahakinbami2959]

Thanks!

[@CodeWithYu]

You’re most welcome! Thanks for the support! ❤️

[@thesungodzzz]

This is a greate video!!!! This is what I expected to see. A greate one, Yu!!! Please keep going. You are inspiring people on this field.

[@CodeWithYu]

Thank you! Will do!

[@SoarSquads]

The view quality is so much better brother keep it up. God bless you and your family

[@CodeWithYu]

Only the best for you!

[@sathyans3852]

This is gold. Amazing work and explanations.👌👌👌

[@danieladeniyi1558]

You are the real MVP. Will sink my teeth into this

[@CodeWithYu]

Great! Don't forget to leave feedback too. Thanks!

[@shusants]

Wonderful content. Thank you so much Yu🙌

[@ravitejapothala977]

Great video! This is what they look in resume and intervie . Not just etl pipeline but Build scalable and fault tolerant one's. Requesting to include kubernetes in the scalable pipelines . Waiting for part 2

[@CodeWithYu]

Great point!

[@DExpertz]

🔥I see Yu post a video, I comment first, then I watch the video. Keep up bro

[@CodeWithYu]

Legend! You know the drill! 🙌😀

[@mohitupadhayay1439]

Great work brother

[@thesungodzzz]

Great video!!!!

[@orafaelgf]

I learn so much with your videos man. Keep going on. If possivel bring to content about Kubernetes ( airflow, spark, strimzi (kafka), data lake, etc.) Tks.

[@CodeWithYu]

Great suggestion!

[@TAUFIQUESEKH]

Very good content. Loved it. Thanks for making this

[@CodeWithYu]

Only the best for you!

[@wiss1998]

Thank you again Youssef !

[@CodeWithYu]

It’s my pleasure Wiss!

[@NgynAn-dg3kp]

love the way you go high level architecture explain with blackboard to the, great 🔥🔥🧑‍🚒

[@CodeWithYu]

Glad you like it! More to come! 🔥

[@souniksadhu4266]

What is the system requirement to run this project in personal laptop?

[@parasagrawal1140]

what is the system requirements? can i make this project in my personel laptop?

[@dhruvingandhi1114]

Can you please tell from where u get file of Docker-compose yml.

[@mohammadbilalniazi5987]

جزاک الله خیرا

[@CodeWithYu]

Ameen. You too brother

[@shubhambhandari431]

Why do we use checkpoint in spark? Can we use group id in kafka to maintain track of records

[@CodeWithYu]

Good question! It’s for fault tolerance. Think of failures during processing or crashes or similar situations, for Kafka to resume from a close reasonable point before failure, checkpoints help with that. Secondly, during stateful processing, checkpoints make the coordination easier. On the other hand, group id is just for coordination of offsets for message delivery. It has nothing to do with checkpointing or fault tolerance. Hope that helps!

[@ATHARVA89]

thank you

[@PRUTHVIRAJ-wp9vu]

Great 👍

[@CodeWithYu]

Thank you! Cheers!

[@nadiiar75]

🔥

[@thirdojohnson4325]

hello boss, just wanted to know if you are available at your convenience for just a question on any of your socials if you dont mind me asking please. Thank you for all you do for us beginners

[@CodeWithYu]

Sure thing, How can I help?

[@flynnryder7823]

Can you please make your screen smaller, thanks

[@bharathdeva8849]

Hello yu. Great content in recent time❤️✨. You are valuing your audience interest, great ❤️✨. I can't see more people doing that. But I see that source code is only for members which 10 dollars. Coming tier 2 city 10 dollars is very high which is 10% of my salary 😢 which I can't afford. Could you consider my request of changing your membership fee?

[@CodeWithYu]

It’ll be available to all youtube members soon

[@shusants]

I agree with @bharathdeva8849. @CodeWithYu, that would be really helpful. Been your subscriber, really loved your content, keep on

[@salvityagi2288]

@@CodeWithYu when it will be available?

[@jay_wright_thats_right]

I like blackboarding but only if you CAN WRITE legibly. 🤣🤣

[@CodeWithYu]

lol sorry… will improve next time. I sometimes wonder if I write those! 😂😂😂😂