Debezium is an open-source platform for log-based change data capture. It reads database transaction logs and emits inserts, updates, and deletes as events for downstream systems.
Debezium is an open-source platform for change data capture. It turns database changes into a stream of events. When a row is inserted, updated, or deleted, Debezium captures the change and sends it downstream.
Most Debezium setups use log-based CDC. Instead of polling tables again and again, Debezium reads the transaction log the database already writes: the MySQL binlog, PostgreSQL write-ahead log, SQL Server change tables, MongoDB change streams or oplog, and similar mechanisms for other databases.
That makes Debezium useful when a warehouse, lakehouse, search index, cache, or another service needs to follow an operational database with low delay.
A Debezium connector connects to a source database, reads changes from the log, and remembers its position. If the connector stops and restarts, it can continue from the last stored position.
Change events have a consistent structure across connectors. An event commonly includes:
before: the row before the change, when available.
after: the row after the change, when available.
op: the operation type, such as create, update, delete, or snapshot read.
source: metadata about the database, table, transaction, log position, and timestamp.
On first start, a connector usually takes a snapshot of existing rows so downstream systems begin with a complete state. After the snapshot, it streams new changes from the log.
Debezium provides source connectors for several databases, including MySQL, MariaDB, PostgreSQL, SQL Server, Oracle, MongoDB, Db2, Cassandra, and others. Support level and connector maturity differ, so production teams should check the current Debezium release and connector documentation for their database version.
The useful part is that events from different connectors follow similar patterns. Downstream applications do not have to learn a completely new event shape for every source.
Kafka Connect
The classic deployment. Debezium connectors run inside Kafka Connect and write change events into Kafka topics, often one topic per table. Kafka stores the stream so several consumers can read it at their own pace.
Debezium Server
A standalone application that streams changes to targets such as Pub/Sub, Kinesis, Pulsar, or other systems without requiring a full Kafka Connect deployment.
Embedded engine
A Java application can embed Debezium and consume change events directly. This gives more control but puts more responsibility in the application.
In Kafka-based setups, Debezium is often paired with a Schema Registry so events can be encoded compactly and schema evolution can be controlled.
Near-real-time analytics. Keep a warehouse or lakehouse close to the operational source without reloading whole tables every night.
Event-driven services. Publish database changes so other services can react to them.
Migrations with less downtime. Load the target once, then use Debezium to catch up with changes until cutover.
Audit and history. Keep a detailed stream of row-level changes for downstream processing.
Debezium is a way to get streaming data from systems that were originally designed as transactional databases rather than event streams.
Many databases and platforms have built-in CDC features. Debezium often uses those features under the hood and turns them into an external stream.
For SQL Server, for example, Debezium requires SQL Server CDC to be enabled and then reads the resulting change records. In Microsoft Fabric, Mirroring can provide managed CDC into OneLake for supported sources. Fabric is easier when your source and target match its supported pattern. Debezium gives more control and broader routing, but you run the infrastructure yourself.
Operational ownership
Kafka, Kafka Connect, connectors, offsets, schema registry, monitoring, and upgrades need real ownership.
At-least-once delivery
After failures or restarts, downstream systems can see duplicate events. Consumers need idempotent processing.
Schema changes
Added columns, renamed columns, and type changes can break consumers unless schema evolution is planned.
Snapshots can be heavy
The initial snapshot may read large tables. Plan it outside peak load or use connector options that fit your source.
Deletes matter
A CDC pipeline that ignores deletes will drift from the source. Decide how tombstones, soft deletes, and hard deletes are represented downstream.
ACID transactions are the four guarantees that keep database changes correct: atomicity, consistency, isolation, and durability. They make s...
Read definitionAnomaly detection automatically flags data points, events, or patterns that do not fit normal behaviour. It can catch odd invoices, machine ...
Read definitionApache Airflow is an open-source workflow orchestrator for batch-oriented data pipelines. You define workflows as Python code, connect tasks...
Read definitionApache Hudi is an open table format for data lakes that makes Parquet files behave like transactional tables. It is strongest where data cha...
Read definitionApache Iceberg is an open table format for large analytical datasets on object storage. It adds snapshots, schema evolution, partition evolu...
Read definition
Seven new Data Panda connectors from June 2026, with practical reporting ideas for stock, finance, ticketing, route planning and operations.
Test data ideas fast with pretotyping. Learn how to validate concepts in days, avoid over-engineering, and build what truly adds value.