Announcing Watermarkless CDC for Estuary Flow

At Estuary Flow, we strive to make capturing Change Data Capture (CDC) streams as seamless, flexible, and low-impact as possible. Today, we’re excited to announce Watermarkless CDC connectors, which allow for capture operations without requiring write access to the source database. This innovation makes Flow more versatile and accessible, especially in environments where write permissions to production databases are a dealbreaker.

Here’s what this means for you across different databases:

MySQL: Watermarkless CDC as the New Standard

For MySQL, watermarkless CDC is a clear win.

• Default Behavior: All new and existing captures in MySQL now operate without watermarks. No configuration is required—it's automatic.
• Read-Only Flexibility: Since watermark writes are no longer required, you can capture CDC streams from read-only replicas. This eliminates the need for write access to your primary database, making it perfect for environments with strict access policies.

PostgreSQL: Power-User Flexibility with "Read-Only Capture"

In PostgreSQL, watermarkless CDC is available as an optional feature, providing flexibility while balancing reliability.

• Default Behavior: Watermark writes are still the default because they serve as a heartbeat for reliable operation and help mitigate PostgreSQL's well-known WAL retention issues.
• Optional "Read-Only Capture": Advanced users can enable a toggle to disable watermark writes, allowing CDC from read-only replicas. However, this comes with caveats:
  • Replication slots on read replicas may get invalidated quickly, especially in setups like Google Cloud SQL.
  • Users must understand the trade-offs and ensure operational reliability.

For most PostgreSQL users, the primary benefit of watermarkless CDC is avoiding the need for write access to production databases. But power users may find additional value in the ability to experiment with read replica setups.

What’s Next for SQL Server?

While watermarkless CDC for SQL Server isn’t ready yet, it’s on our roadmap. We anticipate it will land somewhere between MySQL’s ease of adoption and PostgreSQL’s power-user focus.

• Potential Benefits: When introduced, SQL Server users may be able to reliably capture from read replicas without write permissions, making it an attractive option for secure environments.
• Limitations: Schema management, such as creating or tearing down CDC instances, will still require primary database access.

Why Watermarkless CDC Matters

1. No Write Access Needed: Capture CDC streams without impacting your production database.
2. Read-Only Replica Support: Unlock new deployment flexibility by working with read-only replicas.
3. Reduced Operational Friction: Simplify permissions and make Estuary Flow easier to adopt across teams with strict database access policies.

For MySQL users, this change is automatic and immediate. For PostgreSQL users, you can toggle "Read-Only Capture" in advanced settings to experiment with watermarkless CDC today.

Stay tuned as we continue to improve CDC capabilities across more databases like SQL Server and beyond!

Have questions or want to see it in action?
Let us know—we’d love to help you unlock the full potential of Watermarkless CDC with Estuary Flow.

Send us an email or a message in our Community Slack channel.