Syncing your Database To OpenSearch In Real-Time _Gunnar Morling

Title: Syncing your Database To OpenSearch In Real-Time
Description: You've been tasked with implementing a data streaming pipeline for propagating data changes from your operational Postgres database to a search index in OpenSearch. Data views in OpenSearch should be denormalized for fast querying, and of course there should be no noticeable impact on the production database.
In this session we'll discuss how to build this data pipeline using two popular open-source projects: Debezium for log-based change data capture (CDC) and Apache Flink for stream processing.
* Setting up change data streams with Debezium
* Efficiently building nested data structures from 1:n joins
* Deployment options: Kafka Connect vs. Flink CDC
We'll also touch on some advanced aspects like observability and consistency checks for your realtime data pipeline.
Bio: Gunnar Morling is a software engineer and open-source enthusiast by heart, currently working at Decodable on stream processing based on Apache Flink. In his prior role as a software engineer at Red Hat, he led the Debezium project, a distributed platform for change data capture.
He is a Java Champion and has founded multiple open source projects such as JfrUnit, kcctl, and MapStruct. Gunnar is an avid blogger (morling.dev) and has spoken at various conferences like QCon, Java One, and Devoxx. He lives in Hamburg, Germany.