Change Data Capture and Debezium with Gunnar Morling

[@vishwaspatel4276]

Awesome stuff, thanks for bringing it up Kaivalya. It was fun listening Gunnar(@43:20 "Aha.. actually rollback these transaction", also, relating schema change problem to social problem) and discussion on failure modes was really insightful.

[@TheGeekNarrator]

Thank you Vishwas 🙏🏻😀

[@UCTNX4PiSl496AhgUyGlB5wg]

माहितीपूर्ण आहे. आपण अस म्हणू शकतो का , डेबिझियम एक service आहे ज्यात set of rules configure करुन ते kafka द्वारे stream केले आहेत?

[@TheGeekNarrator]

Yes, in a way. But Debezium isn’t strictly coupled with Kafka.

[@sivaganesh4489]

great talk

[@akashagarwal6390]

without the fear of being judged, how does CDC differ from DB triggers exactly?

[@TheGeekNarrator]

Good question. DB triggers define the action you want to take when some database operation is executed. CDC is more about the event that the operation is executed. The action required depends on the consumer of that event. It de-couples the database from the actions you need to take. Additionally, triggers are limited to other DB operations while CDC enables you to a completely new world.

[@JardaniJovonovich192]

Great Video, appreciate both of your efforts! Let's say, I have a scenario where I need to delete some data(around 1M rows) across some tables that contain both production data and test data without generating CDC for such data. Downstream teams should not get a delete event as the data deleted is a garbage data and of no use to anyone. Is this possible with Debezium?

[@TheGeekNarrator]

I think so, I found documentation around events filtering. debezium.io/documentation/reference/stable/transformations/filtering.html Would be nice to hear what Gunnar has to say about this though. Thanks for your question.

[@GunnarMorling]

@@TheGeekNarrator yes, that's exactly what I would've recommended too. And for cases where this script based filtering transformation is not enough, people could still implement their own custom single message transformation, applying whatever business logic is needed for filtering out unwanted CDC events.

[@akashagarwal6390]

also, isn't CDC just another event-driven approach/system to do things, but just seem to have gotten a name assigned to it for any specific reason?

[@TheGeekNarrator]

They are similar but not the same. Event Driven is a broader concept where components interact with each other with events. Flow of events related to state changes in a system enables loosely coupled architecture, async processesing etc. CDC is a specific way of capturing and replicating database changes. Yes they have events in some way, but cannot be compared directly event drive approach. CDC is typically a part of an event driven architecture. Does that make sense?