7 Best Apache Iceberg Ingestion Tools in 2026: CDC, Streaming & Batch

TL;DR: What Are the Best Apache Iceberg Ingestion Tools?

The best Apache Iceberg ingestion tool depends on where your data originates and how quickly changes need to become available.

• Estuary is best suited to managed CDC, streaming, and batch ingestion from operational databases, SaaS applications, and event streams into Iceberg.
• Qlik Open Lakehouse is a strong option for enterprises that want managed ingestion, transformations, and Iceberg table optimization in AWS.
• Fivetran fits teams that want managed multi-source replication into cloud object storage with Iceberg metadata and table maintenance.
• Confluent Tableflow is particularly well suited to Kafka-centric architectures that need to materialize topics and CDC streams as Iceberg tables.
• Amazon Data Firehose is a strong AWS-native option for continuously delivering streaming records into Iceberg tables on Amazon S3 or S3 Tables.
• Apache Flink provides extensive control for custom, stateful streaming pipelines and supports batch and streaming writes into Iceberg.
• Apache Spark remains a flexible choice for large-scale batch and micro-batch ingestion, transformations, and Iceberg table operations.

For CDC-heavy architectures where source databases, SaaS data, and streaming events need to be unified before delivery to Iceberg, Estuary provides a managed alternative to building and operating the capture, streaming, orchestration, and merge layers separately.

Streaming Data into Apache Iceberg: Tools for a Scalable Data Lakehouse

Apache Iceberg is an open table format, not a data ingestion system. Iceberg provides capabilities such as ACID transactions, schema evolution, hidden partitioning, and snapshot-based table management, but teams still need a pipeline to move data from operational systems, event streams, and applications into Iceberg tables.

For production pipelines, simply writing Parquet files to object storage is not enough. The ingestion layer may also need to:

• Capture inserts, updates, and deletes correctly.
• Commit changes to Iceberg tables reliably.
• Handle schema changes without breaking downstream workloads.
• Work with the required Iceberg catalog.
• Control commit frequency and data freshness.
• Prevent excessive small files and snapshots from degrading query performance.
• Recover safely from pipeline or compute failures.

These requirements become especially important for change data capture (CDC) and streaming workloads because frequent writes can increase the number of snapshots, metadata files, and small data files that need to be maintained. Apache Iceberg's own Spark documentation recommends tuning commit frequency and regularly compacting files for streaming tables.

The seven tools below take different approaches. Some provide fully managed ingestion, while others give engineering teams frameworks for building their own Iceberg pipelines.

Apache Iceberg Ingestion Tools Compared

No single tool is best for every architecture. Teams should evaluate Iceberg ingestion tools based on their source systems, required data freshness, CDC requirements, catalog compatibility, table-maintenance strategy, and how much pipeline infrastructure they want to operate themselves.

How We Evaluated These Apache Iceberg Ingestion Tools

We evaluated each tool based on how well it supports production Apache Iceberg ingestion rather than simply whether it can write files to object storage.

The main criteria were:

• CDC support: Whether the tool can capture and apply inserts, updates, and deletes from operational databases.
• Streaming and batch support: Whether it supports continuous ingestion, batch workloads, or both.
• Iceberg-native writes: How the tool creates data files, commits Iceberg metadata, and maintains table state.
• Schema evolution: How changes to source schemas are propagated without breaking downstream pipelines.
• Catalog compatibility: Support for Iceberg REST catalogs, AWS Glue, Amazon S3 Tables, Snowflake Open Catalog, or other catalog implementations.
• Table maintenance: How the platform handles small files, snapshots, metadata growth, compaction, and other ongoing Iceberg maintenance.
• Operational overhead: How much infrastructure, orchestration, and pipeline code a team needs to deploy and maintain.
• Source coverage: Whether the tool primarily serves Kafka and event streams or can also ingest from databases, SaaS applications, APIs, and other systems.

We reviewed current first-party product documentation and Apache Iceberg documentation for the technical capabilities discussed below. Where tools take fundamentally different approaches, we focus on their best-fit use cases rather than assigning a single overall winner.

What to Look for in an Apache Iceberg Ingestion Tool

Choosing an Iceberg ingestion tool involves more than comparing connector counts. The ingestion architecture affects data freshness, correctness, infrastructure requirements, and the ongoing maintenance of Iceberg tables.

1. CDC and Row-Level Changes

If the source is an operational database, check whether the tool can preserve inserts, updates, and deletes rather than only appending new records.

This matters because CDC pipelines need to translate source-level changes into the appropriate Iceberg table operations while maintaining a consistent representation of the source.

For databases that change frequently, prefer tools with native CDC support rather than repeatedly scanning entire source tables.

2. Streaming vs. Micro-Batch Ingestion

“Real-time Iceberg ingestion” can describe several different architectures.

Some tools continuously capture changes but commit them to Iceberg at configurable intervals. Others use streaming frameworks such as Apache Flink or Spark Structured Streaming to write frequent checkpoints or micro-batches.

The important metric is therefore not only how quickly a source change is captured, but how quickly that change becomes queryable in the Iceberg table.

Teams should choose the commit frequency based on freshness requirements, compute cost, and table-maintenance overhead.

3. Small Files and Table Maintenance

Frequent commits can create large numbers of small data files, manifests, metadata files, and snapshots.

Apache Iceberg recommends regular maintenance for frequently updated tables, including:

• Compacting small data files.
• Expiring old snapshots.
• Cleaning unused metadata.
• Rewriting manifests when necessary.

These operations help keep query performance and metadata growth under control.

See the official Apache Iceberg maintenance documentation for more details.

4. Schema Evolution

Source schemas change over time. Columns may be added, renamed, or have compatible type changes.

A production ingestion tool should provide a predictable way to propagate supported schema changes into Iceberg without requiring teams to manually rebuild pipelines or tables for every source-side change.

Iceberg itself supports schema evolution while preserving field identity, but the ingestion platform still determines how source schema changes are detected and applied.

5. Iceberg Catalog Support

Iceberg tables depend on a catalog to coordinate table metadata.

Depending on the architecture, teams may use:

• Iceberg REST Catalog
• AWS Glue Data Catalog
• Amazon S3 Tables
• Snowflake Open Catalog
• Apache Polaris
• Other Iceberg-compatible catalogs

Before selecting an ingestion platform, confirm that it works with the catalog used by your storage and query engines.

6. Infrastructure and Operational Complexity

Apache Spark, Apache Flink, and Kafka Connect provide significant flexibility, but engineering teams are generally responsible for deploying, monitoring, scaling, and recovering the underlying workloads.

Managed platforms shift more of this responsibility to the vendor.

The right choice depends on whether your team prioritizes maximum control or lower operational overhead.

7. Source Coverage

The best ingestion architecture also depends on where the data originates.

A Kafka-native solution can be ideal when all relevant data already flows through Kafka. However, organizations ingesting from PostgreSQL, MySQL, SQL Server, SaaS applications, and event streams may benefit from a platform that supports those sources directly rather than introducing Kafka purely as an intermediate ingestion layer.

7 Best Tools to Stream and Ingest Data into Apache Iceberg

1. Estuary

Best for: Managed CDC, streaming, and batch ingestion from databases, SaaS applications, and event streams into Apache Iceberg.

Estuary is a real-time data integration platform that supports change data capture (CDC), streaming, and batch ingestion. It is particularly useful for teams that need to continuously capture database changes and materialize them into Iceberg without building separate CDC, streaming, and orchestration pipelines.

For Apache Iceberg, Estuary captures source changes continuously and merges accumulated updates into Iceberg tables using Spark on AWS EMR Serverless. Teams can configure the sync frequency to balance data freshness, compute cost, and Iceberg table maintenance.

Key capabilities:

• Native CDC from databases such as PostgreSQL, MySQL, and SQL Server.
• Support for inserts, updates, and deletes.
• Configurable Iceberg materialization frequency.
• Support for Iceberg REST catalogs including AWS Glue, Amazon S3 Tables, and S3-backed compatible catalogs.
• Schema-evolution workflows and Iceberg table-maintenance options.

Best fit: Estuary stands out when data originates across operational databases, SaaS systems, and event streams rather than already living in Kafka.

Production evidence: Prodege uses Estuary with Apache Iceberg on Amazon S3 and reported a 60% reduction in replication costs and 30% lower Snowflake costs.

Limitation: The current general-purpose Iceberg materialization uses Amazon S3 staging and AWS EMR Serverless, so teams using fully Azure- or GCP-native Iceberg architectures should confirm compatibility.

Related Articles on Using Estuary to Ingest Data into Apache Iceberg:

• Steps to Load Data Into Iceberg with Estuary
• Load Data From Redshift to Iceberg
• Load Data From BigQuery to Iceberg
• Load Data From Kafka to Iceberg
• Load Data from Postgres to Iceberg

2. Qlik Open Lakehouse

Best for: Enterprises building managed Apache Iceberg lakehouses on AWS with CDC, streaming, and batch ingestion.

Qlik Open Lakehouse is an Apache Iceberg-based service within Qlik Talend Cloud. It supports ingestion from databases, SaaS applications, and streaming sources into query-ready Iceberg tables running in the customer's AWS environment.

For database and SaaS sources, Qlik supports change data capture (CDC) after the initial load. It also supports high-throughput streaming ingestion from sources such as Apache Kafka, Amazon Kinesis, and Amazon S3, with changes continuously applied to Iceberg tables.

Key capabilities:

• CDC, batch, and streaming ingestion into Apache Iceberg.
• Kafka, Kinesis, and S3 streaming sources.
• Schema evolution and transformations.
• Iceberg table optimization and snapshot-management controls.
• AWS Glue Data Catalog integration.
• Ability to mirror Iceberg datasets into Snowflake or Amazon Redshift.

Best fit: Qlik is a strong choice for enterprises already using Qlik Talend Cloud or teams that want ingestion, transformation, optimization, and governance within a managed AWS-based Iceberg architecture.

Limitation: Qlik Open Lakehouse currently requires an AWS-based environment and AWS Glue Data Catalog, making it less suitable for teams looking for a cloud-neutral Iceberg ingestion architecture.

3. Fivetran

Best for: Teams that want managed replication from many data sources into Iceberg tables stored in their own cloud object storage.

Fivetran Managed Data Lake Service writes source data as Parquet files to Amazon S3, Azure Data Lake Storage, or Google Cloud Storage while automatically creating and maintaining Apache Iceberg metadata.

Fivetran manages the catalog and ongoing table maintenance, reducing the operational work required to keep Iceberg tables queryable. Its default Fivetran Catalog is based on Apache Polaris and implements the Iceberg REST protocol, while integrations such as AWS Glue and BigLake Metastore are also supported.

Key capabilities:

• Managed ingestion from databases, SaaS applications, and other supported sources.
• Automatic Iceberg metadata and catalog management.
• Support for Amazon S3, ADLS, and Google Cloud Storage.
• Automatic schema-change handling.
• Ongoing table optimization and maintenance.
• Iceberg REST catalog based on Apache Polaris.

Best fit: Fivetran is a strong option for teams prioritizing broad source coverage and low operational overhead, particularly when they already use Fivetran for data replication.

Limitation: Fivetran's Managed Data Lake Service is designed primarily around scheduled source synchronization rather than low-latency event streaming. It also currently does not support Iceberg position deletes or transformations within the Managed Data Lake Service itself.

4. Confluent Tableflow

Best for: Kafka-centric architectures that want to materialize streaming topics and CDC feeds directly as Apache Iceberg tables.

Confluent Tableflow is a managed Confluent Cloud service that converts Kafka topics into Apache Iceberg or Delta Lake tables. It automatically handles schema conversion, schema evolution, catalog publishing, CDC materialization, and table maintenance.

For CDC workloads, Tableflow can materialize Debezium change streams from sources such as PostgreSQL, MySQL, and SQL Server after those changes have first been captured into Kafka. It also automatically compacts small files created by continuous streaming workloads.

Key capabilities:

• Materializes Kafka topics as Apache Iceberg tables.
• Supports CDC streams and upsert-style workloads.
• Automatic schema evolution and Parquet conversion.
• Automatic file compaction and table maintenance.
• Supports Confluent-managed or customer-owned cloud storage.
• Integrates with AWS Glue, Iceberg REST catalogs, and Snowflake Open Catalog/Apache Polaris.

Best fit: Tableflow is especially strong when Kafka or Confluent Cloud is already the central event backbone and teams want to expose streaming data to lakehouse query engines without building a custom Kafka-to-Iceberg pipeline.

Limitation: Tableflow is primarily a Kafka-to-table solution. Database CDC still requires changes to first enter Kafka through a compatible CDC connector such as Debezium, making it less direct for teams that do not otherwise need Kafka.

5. Amazon Data Firehose

Best for: AWS-native teams that need a serverless way to continuously deliver streaming data into Apache Iceberg tables.

Amazon Data Firehose can deliver streaming records directly into Apache Iceberg tables stored in Amazon S3 or Amazon S3 Tables. It can route records from a single stream to multiple Iceberg tables and apply insert, update, and delete operations using configured unique keys.

Firehose is fully managed and serverless, making it attractive for teams already using AWS services such as Amazon Kinesis, Amazon MSK, CloudWatch, or other supported streaming sources.

Key capabilities:

• Direct streaming writes into Apache Iceberg.
• Insert, update, and delete operations.
• Exactly-once delivery to Iceberg tables.
• Support for Amazon S3 and Amazon S3 Tables.
• AWS Lake Formation integration for fine-grained access control.
• Serverless operation without managing Spark or Flink clusters.

Best fit: Firehose is a strong choice when streaming data already lives within AWS and the goal is to land it directly in Iceberg with minimal infrastructure to operate.

Limitation: Firehose is primarily a streaming delivery service rather than a broad CDC and SaaS ingestion platform. Database CDC generally requires an upstream change-capture system, and its Iceberg implementation is closely tied to AWS storage, catalog, and IAM services.

6. Apache Flink

Best for: Engineering teams that need fine-grained control over low-latency, stateful streaming pipelines into Apache Iceberg.

Apache Flink is an open-source stream-processing framework with native Apache Iceberg integration. Iceberg supports both batch and streaming writes from Flink through the DataStream and Table APIs, making Flink a strong option for custom event-processing and CDC pipelines.

The Iceberg Flink sink supports exactly-once semantics and can perform upserts into Iceberg v2 tables when primary-key or identifier fields are defined.

Key capabilities:

• Native batch and streaming writes into Iceberg.
• Exactly-once sink semantics.
• Upserts into Iceberg v2 tables.
• SQL and DataStream APIs.
• Dynamic routing to multiple Iceberg tables.
• Fine-grained control over checkpoints, write distribution, and file sizing.

Best fit: Flink is a strong choice when teams need custom transformations, stateful processing, or very low-latency streaming and are comfortable operating Flink infrastructure.

Limitation: Flink provides more control than a managed ingestion platform, but it also requires teams to manage jobs, checkpoints, scaling, failure recovery, source connectors, and Iceberg maintenance. Snapshot expiration and orphan-file cleanup must also be coordinated carefully with active Flink streaming jobs.

7. Apache Spark

Best for: Teams that need flexible batch and micro-batch processing, transformations, and custom Iceberg pipelines.

Apache Spark integrates closely with Apache Iceberg for large-scale batch processing and Structured Streaming workloads. Spark can read from and write to Iceberg tables, perform MERGE, UPDATE, and DELETE operations, and support complex transformations before data is committed.

For streaming workloads, Spark Structured Streaming uses a micro-batch model rather than processing each record as an independent Iceberg commit. This gives teams flexibility over trigger intervals and commit frequency, but frequent writes can create additional snapshots, metadata files, and small data files that require regular maintenance.

Key capabilities:

• Batch and micro-batch writes into Apache Iceberg.
• MERGE, UPDATE, and DELETE support.
• Complex SQL and DataFrame transformations.
• Broad Iceberg catalog support.
• Configurable streaming trigger and commit frequency.
• Iceberg maintenance procedures for compaction and snapshot management.

Best fit: Spark is a strong choice for engineering teams that already operate Spark or need extensive transformation logic before writing data into Iceberg.

Limitation: Spark provides considerable flexibility, but teams are responsible for operating compute, orchestrating jobs, handling failures, and managing Iceberg maintenance. For CDC, Spark also typically needs an upstream system to capture database changes before they can be processed and merged into Iceberg.

Other Apache Iceberg Ingestion Options Worth Considering

The seven tools above cover the most common managed, streaming, CDC, and batch ingestion patterns, but several other platforms are worth considering for specific Iceberg architectures.

Kafka Connect

Best for: Kafka-native teams that want a lightweight, configurable Iceberg sink.

The official Apache Iceberg Sink Connector for Kafka Connect writes Kafka records directly into Iceberg tables and supports exactly-once delivery, automatic table creation, schema evolution, and multi-table fan-out.

Kafka Connect is a strong option when data already flows through Kafka and teams are comfortable operating Kafka Connect workers. However, database CDC generally still requires a source connector such as Debezium before records reach the Iceberg sink.

Dremio

Best for: Teams focused on Iceberg catalog management, governance, query performance, and lakehouse interoperability.

Dremio uses Apache Iceberg as its native table format and provides an Iceberg REST-compatible Open Catalog built on Apache Polaris. It supports read/write access from engines such as Spark and Flink and can automate Iceberg maintenance tasks including compaction and vacuum operations.

Dremio is particularly valuable as a catalog and lakehouse management layer, rather than as a general-purpose CDC ingestion platform.

RisingWave

Best for: Streaming SQL pipelines that transform and continuously deliver changing data into Iceberg.

RisingWave is a streaming database that can sink continuously changing data into Apache Iceberg using append-only or upsert modes. Its Iceberg sink supports updates and deletes by primary key, exactly-once delivery, configurable commit intervals, schema evolution, and automated table maintenance.

It is especially useful when teams need to perform streaming joins, aggregations, or transformations before materializing results into Iceberg.

How to Choose the Right Apache Iceberg Ingestion Tool

The right Iceberg ingestion tool depends primarily on where your data comes from, how fresh it needs to be, and how much infrastructure your team wants to operate.

Choose Estuary if your data comes from operational databases and SaaS systems

Estuary is a strong fit when you need to capture changes from databases such as PostgreSQL, MySQL, or SQL Server alongside SaaS and streaming sources, then continuously materialize that data into Iceberg.

This is especially useful when you want managed change data capture (CDC) without introducing Kafka, separate CDC connectors, and custom Spark orchestration purely for ingestion.

Choose Confluent Tableflow or Kafka Connect if your data already lives in Kafka

If Kafka is already your central event backbone, Confluent Tableflow provides a managed way to materialize Kafka topics into Iceberg, while the Apache Iceberg Kafka Connect sink offers an open-source approach with more infrastructure ownership.

Introducing Kafka only to reach Iceberg, however, may add unnecessary complexity when source data primarily lives in operational databases or SaaS applications.

Choose Amazon Data Firehose for AWS-native event ingestion

Amazon Data Firehose is a strong option when streaming data already flows through AWS and needs to land in Iceberg tables on Amazon S3 or S3 Tables.

It offers a serverless operational model, but database CDC generally requires an upstream capture layer.

Choose Apache Flink for custom stateful streaming

Apache Flink is well suited to teams that need custom transformations, joins, stateful stream processing, or fine control over streaming behavior before writing into Iceberg.

The tradeoff is greater operational responsibility for jobs, checkpoints, scaling, and table maintenance.

Choose Apache Spark for batch and micro-batch workloads

Apache Spark remains a strong option for large-scale batch processing, complex transformations, and micro-batch ingestion into Iceberg.

It is particularly attractive for teams that already operate Spark infrastructure, but frequent streaming commits require careful management of snapshots, metadata, and small files.

Choose Fivetran or Qlik when managed lakehouse ingestion is the priority

Fivetran is a good fit for teams that want broad connector coverage and managed Iceberg metadata with low operational overhead.

Qlik Open Lakehouse is better suited to enterprises building managed AWS-based Iceberg lakehouses that also need CDC, streaming, transformations, and table optimization.

A Simple Decision Rule

If you are choosing quickly:

• Database CDC + SaaS + streaming sources: Estuary
• Kafka-native pipelines: Confluent Tableflow or Kafka Connect
• AWS-native streaming: Amazon Data Firehose
• Custom low-latency stream processing: Apache Flink
• Batch and micro-batch processing: Apache Spark
• Managed multi-source lake ingestion: Fivetran
• Managed enterprise AWS lakehouse: Qlik Open Lakehouse

Regardless of the tool, also evaluate catalog compatibility, update/delete handling, schema evolution, commit frequency, and table maintenance. Frequent Iceberg writes can create additional snapshots, metadata, and small files, so compaction and snapshot management should be part of the architecture rather than an afterthought.

Final Thoughts

There is no single best Apache Iceberg ingestion tool for every architecture. The right choice depends on your source systems, freshness requirements, CDC needs, Iceberg catalog, and how much infrastructure your team wants to operate.

For database CDC and multi-source ingestion, Estuary is a strong managed option. Kafka-centric teams may prefer Confluent Tableflow or Kafka Connect, while Amazon Data Firehose fits AWS-native streaming. Flink and Spark provide more control for teams that want to build and operate custom streaming or batch pipelines.

Whichever approach you choose, evaluate not only ingestion speed but also updates and deletes, schema evolution, catalog compatibility, commit frequency, and ongoing Iceberg table maintenance.

Ready to build an Apache Iceberg pipeline? Explore Estuary's Apache Iceberg integration or start building for free.