Skip to main content

Oracle Xstream CDC Source Connector

Overview and Features of Oracle XStream CDC Source Connector for Confluent Cloud​

The fully-managed Oracle XStream CDC Source connector for Confluent Cloud captures all changes made to rows in an Oracle database and represents the changes as change event records in Apache Kafka® topics. The connector uses Oracle’s XStream API to read changes from the database redo log.

Oracle XStream is a set of components and APIs in Oracle database that enables client applications, such as the connector, to receive changes from an Oracle database.

The connector leverages XStream Out to capture both Data Manipulation Language (DML) and Data Definition Language (DDL) changes from the database redo log. When XStream Out is used, a capture process captures changes made to an Oracle database, converts the changes into Logical Change Records (LCRs), and sends the LCRs to an outbound server. The outbound server then sends the LCRs to the connector.

The connector is built using the Debezium and Kafka Connect frameworks.

Features​

The Oracle XStream CDC Source connector provides the following features:

Oracle End User Terms​

The Oracle XStream CDC source connector uses Oracle’s XStream API technology to read data from the logs. XStream technology is licensed as part of the Oracle GoldenGate family. Confluent has an agreement with Oracle that allows Confluent to use this technology with its products. Therefore, users do not need to buy additional licenses from Oracle to use XStream with this connector.

Oracle XStream CDC Source Connector for Confluent Cloud​

The connector can be configured to capture changes from a subset of tables in a database by using an include regular expression to match the table identifiers. It can also be configured to not capture tables that match a separate exclude regular expression.

Oracle versions​

The connector is compatible with the following Oracle versions:

  • Oracle 19c Enterprise Edition
  • Oracle 19c Standard Edition
  • Oracle 21c Enterprise Edition
  • Oracle 21c Standard Edition

The connector supports Oracle Exadata.

The connector supports Oracle Database 19c using the non-CDB architecture on Amazon RDS for Oracle.

Oracle XStream CDC Source Connector for Confluent Cloud | Confluent Documentation

Oracle Database Prerequisites for Oracle XStream CDC Source Connector for Confluent Cloud​

Working with Amazon RDS for Oracle​

The connector supports Oracle Database 19c using the non-CDB architecture on Amazon RDS for Oracle. It does not support CDBs on Amazon RDS for Oracle.

The connector also does not support Amazon RDS Custom for Oracle, including both CDB and non-CDB architectures.

This section only highlights the differences in the prerequisite setup steps specific to Amazon RDS for Oracle.

Because Amazon RDS for Oracle is a managed service, you do not have access to a user with SYSDBA privileges. Instead, use the master user account to perform any administrative tasks described in this documentation that requires SYSDBA privileges.

Oracle Database Prerequisites for Oracle XStream CDC Source Connector for Confluent Cloud | Confluent Documentation

Stream Changes from Oracle Database to Confluent Cloud​

The connector creates change events for database changes and sends them to Kafka topics. A change event includes a key and a value, with their structures based on the schema of the table where the change occurred.

Every message in key and value consists of two components: a schema and a payload. The schema defines the structure of the payload, and the payload holds the actual data.

Stream Changes from Oracle Database to Confluent Cloud | Confluent Documentation

Signals and Actions for Oracle XStream CDC Source Connector for Confluent Cloud​

The connector offers a signaling mechanism to trigger specific actions, such as performing an ad-hoc snapshot of a table.

You can send signals through one or more channels, such as a database table or a Kafka topic. To configure which channels are enabled, use the signal.enabled.channels configuration property.

Signals and Actions for Oracle XStream CDC Source Connector for Confluent Cloud | Confluent Documentation

Challenges with LogMiner connector​

  1. Performance: LogMiner is single threaded. It was designed as a diagnostic tool for reading the database redo logs.
  2. Long running transaction: As the transaction ages, the connector has to mine a larger SCN range leading to higher resource utilization on the database and increased latencies.
  3. Large transaction: High memory usage that increases with the size of the transaction. This could cause the connector to fail due to insufficient memory on the Connect worker.
  4. Connector restarts: When the connector restarts, it needs to start mining from an SCN that corresponds to the start SCN of the oldest open transaction. If the archived redo log for this SCN is no longer available, the restart fails.
  5. Schema change handling: Requires the database administrator to extract the log miner dictionary to the redo log files when a DDL statement is executed to change the schema.
  6. Oracle Real Application Cluster support: A 3 second delay is implemented in the mining process to ensure there is no data loss at high load.