Top 10 Query Engines for Apache Iceberg: A Complete Comparison

Apache Iceberg is a high-performance table format designed for large analytic datasets. It introduces hidden partitioning, schema evolution, snapshot isolation, and time-travel queries. As more organizations move to data lake-based architectures, they need query engines that fully leverage Iceberg’s transaction model, schema evolution, and partitioning optimizations.

This post will compare 10 query engines for Apache Iceberg across various technical dimensions. When evaluating this data lakehouse architecture component, we will focus only on the most essential aspects.

1. Integration Approach (Connector vs. Native Support)
   
2. Read/Write Capabilities (Batch, Streaming, or Both)
   
3. Metadata Handling (Snapshot management, partition evolution)
   
4. Performance Characteristics (Concurrency, scalability, resource usage)
   
5. Limitations and Known Drawbacks

The goal is to help you, the data engineer, make an informed choice when picking a query engine for Iceberg-based data transformations or analytics.

The Role of a Query Engine in a Lakehouse

Before we begin the comparison, let’s clarify a few things about the components of a lakehouse.

An iceberg-based data lakehouse combines the scalability and flexibility of a data lake with the structure and governance of a data warehouse. This allows organizations to store and process large volumes of data cost-effectively and efficiently while performing complex queries and analytics.

The general high-level architecture of an iceberg-based data lakehouse typically consists of the following components:

• Data Lake: The data lake is the central storage repository for all data, regardless of format or structure. This includes raw data, processed data, and curated datasets.
  
• Iceberg Tables: Iceberg tables organize and manage data within the data lake. They provide a consistent metadata layer that abstracts away the underlying storage details and enables efficient data access and querying.
  
• Catalog: The catalog is the critical component that keeps table definitions and metadata consistent and discoverable across all the engines, processing tools, and analytics services in your data lakehouse.
  
• Query Engines: Query engines interact with the data stored in Iceberg tables. They provide an SQL-like interface for querying the data and can leverage Iceberg's features, such as snapshot isolation, time travel, and partitioning optimizations.
  
• Data Processing Tools: Data processing tools transform and prepare data for analysis. They can perform various tasks like data cleaning, feature engineering, and model training.
  
• BI and Analytics Tools: BI and analytics tools visualize and analyze the data stored in the data lakehouse. These tools can create reports, dashboards, and other visualizations to help users understand the data and make informed decisions.

Query engines play a crucial role in lakehouse architecture. They provide a means to access and analyze the data stored in Iceberg tables. They allow users to perform complex queries and analytics on large volumes of data in a scalable and efficient manner.

Now that the basics are out of the way, let’s examine the options more deeply.

Summary Table: 10 Query Engines for Iceberg

Below is a concise table summarizing the main technical aspects of each engine’s Iceberg support:

Detailed Analysis of Each Query Engine

1. Apache Spark

Apache Spark is a widely adopted open-source distributed computing engine for large-scale data processing. It is known for its speed, scalability, and widespread compatibility. Spark's integration with Apache Iceberg offers comprehensive support for Iceberg's features, such as hidden partitioning, schema evolution, snapshot isolation, and time-travel queries.

Integration Approach:

• Spark has first-class support for Iceberg through the iceberg-spark runtime package.
  
• You can load it via Spark session configuration:

spark

plaintext
  .read
  .format("iceberg")
  .load("database.iceberg_table")

Read/Write Capabilities:

• Supports both batch (Spark SQL, DataFrame API) and structured streaming.
  
• Example batch write:

plaintext
df.write
  .format("iceberg")
  .mode("append")
  .save("database.iceberg_table")

• For streaming, you can use readStream and writeStream methods with the Iceberg format.

Metadata Handling:

• Full support for Iceberg snapshots, time travel, and partition evolution.
  
• Time-travel query example:

plaintext
SELECT * FROM database.iceberg_table VERSION AS OF 1234567890123

Performance Characteristics:

• Known for large-scale transformations and ML pipelines.
  
• Shuffle operations can become a bottleneck if not appropriately managed (e.g., using partition pruning or broadcast joins).

Drawbacks:

• Spark’s overhead (executing large DAGs) can cause latency for small or highly frequent updates.
  
• Managing cluster resources can be complex, especially with dynamic resource allocation.

2. Apache Flink

Apache Flink is a popular open-source distributed stream processing framework known for its low latency, high throughput, and stateful computations. It excels in real-time analytics, event-driven applications, and stream processing pipelines. Flink's integration with Apache Iceberg provides a robust solution for scalable data management and querying on large-scale datasets.

Integration Approach:

• Flink offers a native flink-iceberg connector.
  
• You can use the Flink SQL API or DataStream API to interact with Iceberg tables.

Read/Write Capabilities:

• Supports continuous streaming reads and writes for real-time data ingestion.
  
• Batch mode is also supported by running Flink jobs in bounded mode.
  
• Example SQL:

plaintext
CREATE TABLE iceberg_table (
  id INT,
  data STRING
) WITH (
  'connector'='iceberg',
  'catalog-type'='hive',
  'uri'='thrift://hive-metastore:9083',
  'warehouse'='s3://bucket/warehouse'
);

INSERT INTO iceberg_table SELECT id, data FROM source_table;

Metadata Handling:

• Supports Iceberg snapshots, partition evolution, and CDC (Change Data Capture) patterns.
  
• Exactly-once guarantees depend on proper checkpoint configurations.

Performance Characteristics:

• Ideal for streaming pipelines with moderate to high throughput.
  
• Checkpoint intervals and state backends can affect latency and throughput.

Drawbacks:

• Requires careful tuning of checkpoint intervals for consistent state.
  
• More complex to operate compared to purely batch-focused engines if real-time ingestion is not needed.

3. Trino

Trino, formerly PrestoSQL, is a distributed SQL query engine that supports batch and real-time analytics. It is designed for fast, interactive querying of large-scale datasets and is widely used for business intelligence, data exploration, and ad hoc analysis. Trino's key strengths include its ability to handle complex queries efficiently, scale horizontally to meet increasing data volumes, and integrate with various data sources and storage systems.

Integration Approach:

• Trino ships with an Iceberg connector plugin that can be enabled in its configuration.
  
• Typically uses a Hive metastore for table metadata, though catalog options exist (e.g., AWS Glue).

Read/Write Capabilities:

• Primarily batch/interactive queries via SQL.
  
• Example usage:

plaintext
CREATE TABLE iceberg_catalog.db.iceberg_table (
  id VARCHAR,
  timestamp TIMESTAMP,
  value DOUBLE
) 
WITH (partitioning = ARRAY['bucket(id, 8)']);

• Writes are supported (INSERT, CREATE TABLE AS SELECT), but no streaming ingestion.

Metadata Handling:

• Fully leverages Iceberg metadata features: snapshots, time-travel, and hidden partitioning.
  
• Offers SQL syntax for time travel:

plaintext
SELECT * FROM iceberg_catalog.db.iceberg_table FOR TIMESTAMP AS OF '2024-01-01 00:00:00';

Performance Characteristics:

• Optimized for interactive analytics, federated queries, and concurrency across multiple data sources.
  
• Memory usage can spike if queries are not well-partitioned or if large joins occur.

Drawbacks:

• Lacks real-time streaming capabilities.
  
• Configuration can be non-trivial for enterprise deployments with many connectors.

4. MotherDuck

MotherDuck is a cloud-native analytics engine built around DuckDB. It provides a serverless, low-latency environment for data analysis using familiar SQL syntax. By leveraging DuckDB’s in-process, columnar execution model, MotherDuck is well-suited for both interactive analytics and smaller-scale data workflows without the overhead of traditional data warehousing systems.

Integration Approach

• Emerging Connector: MotherDuck’s integration with Iceberg is evolving, aiming to allow direct read/write operations against Iceberg tables.
  
• Lightweight Setup: Unlike Hadoop-based systems, MotherDuck requires no heavy cluster configuration. It runs efficiently as a managed service, allowing users to focus on analysis rather than infrastructure.

Read/Write Capabilities

• SQL-Driven: Users interact with data via standard SQL, benefiting from DuckDB’s straightforward query syntax.
  
• Batch and Interactive: While it can support batch-oriented workflows, MotherDuck’s real strength is in fast, interactive queries that return results in seconds.

Example

plaintext
-- query data
SELECT count(*) 
FROM iceberg_scan('path-to-iceberg-folder', 
allow_moved_paths=true);

-- query metadata
SELECT * 
FROM iceberg_metadata('path-to-iceberg-folder', 
allow_moved_paths=true);

-- query snapshots
SELECT * 
FROM iceberg_snapshots('path-to-iceberg-folder');

Metadata Handling

• Catalog Integration: Support is currently identical to what the underlying DuckDB extension supports.
  
• Minimal Overhead: Since MotherDuck is serverless, metadata is fetched on-demand, reducing the need for persistent cluster resources.

Performance Characteristics

• Optimized for Ad Hoc: Built on DuckDB’s vectorized engine, MotherDuck excels at quick, iterative analytics over moderate-to-large datasets.
  
• Concurrency: Ongoing improvements aim to handle higher concurrency while maintaining sub-second response times.

Drawbacks

• Maturing Support: As of now, Iceberg integration is in its early stages, so advanced features like partition evolution or full snapshot isolation may be limited.
  
• Ecosystem Reach: While the core DuckDB engine is powerful, some enterprise features—common in more established data warehouse platforms—are still on the roadmap.

MotherDuck stands out for easy, interactive analytics, and it aims to combine the flexibility of DuckDB with Iceberg’s modern table format.

5. Dremio

Dremio is an open-source distributed SQL query engine for high-performance analytics on large-scale datasets. It enables interactive querying of data stored in various data sources, including cloud object storage, Hadoop Distributed File System (HDFS), and relational databases. Dremio's key strengths lie in its ability to handle complex queries efficiently, support real-time data ingestion and processing, and provide a unified view of data across multiple sources.

Integration Approach:

• Dremio has native support for Iceberg with an integrated connector.
  
• Creates a logical view of the data for interactive SQL queries and BI consumption.

Read/Write Capabilities:

• Focuses on read-optimized queries.
  
• Though not as commonly used for streaming, writing is possible through CTAS or reflection-based transformations.
  • Example:

plaintext
CREATE TABLE dremio_space."iceberg_db.iceberg_table" AS 
SELECT id, SUM(value) as total_value
FROM dremio_space."iceberg_db.source_table"
GROUP BY id;

Metadata Handling:

• Full support for partition evolution, hidden partitioning, and snapshot-based reads.

Performance Characteristics:

• Columnar execution engine with accelerations (reflections).
  
• Excels at interactive analytics across various data sources.

Drawbacks:

• Memory requirements can be high in large clusters.
  
• Mainly optimized for read queries, heavy write use cases might need additional configuration.

6. Snowflake

Snowflake is a cloud-based, fully managed data warehouse and business intelligence service designed for fast, scalable, and secure data analysis. It offers a range of features, including an SQL interface, support for semi-structured data, and built-in data governance and security features.

While Snowflake is a fully-fledged data warehouse, with its latest advances around the Iceberg ecosystem, it can be used as a dedicated compute engine, with all the Snowflake features that data practitioners love included.

Integration Approach:

• Snowflake integrates with Iceberg as an external table.
  
• Iceberg data typically resides in cloud storage (S3, ADLS, GCS), and Snowflake queries it via external location references.

Read/Write Capabilities:

• Primarily read-oriented with external tables.
  
• Write operations may require staging data in Snowflake, copying it to Iceberg-managed locations, and then refreshing metadata.
  
• Example:

plaintext
CREATE EXTERNAL TABLE my_iceberg_table
WITH LOCATION = '@my_external_stage/iceberg/db/table'
FILE_FORMAT = (TYPE = PARQUET);
SELECT * FROM my_iceberg_table;

Metadata Handling:

• Snowflake external tables do not directly recognize time travel in Iceberg.
  
• Additional steps needed to refresh partitions or reflect new snapshots.

Performance Characteristics:

• Suitable for ad-hoc analysis if you prefer Snowflake’s elasticity and skip heavy ETL outside.
  
• Network and I/O can become a bottleneck due to external data location.
  
• Iceberg external tables performance on par with native tables.

Drawbacks:

• External table sync steps can complicate pipelines.

7. DuckDB

DuckDB is an open-source, in-memory SQL database designed for fast analytics on large datasets. It is known for its high performance and ability to handle complex queries efficiently. DuckDB is often used for data exploration, interactive querying, and real-time analytics.

Integration Approach:

• DuckDB has an early-stage Iceberg extension allows data to be read from Iceberg tables.
  
• Aims to provide local analytics capabilities.

Read/Write Capabilities:

• Batch-oriented for local or embedded analytics.
  
• No write support.
  
• Example usage:

plaintext
SELECT count(*)
FROM iceberg_scan('data/iceberg/lineitem_iceberg', allow_moved_paths = true);

Metadata Handling:

• Basic snapshot reading might be possible, but advanced features (time travel, partition evolution) are still limited or experimental.

Performance Characteristics:

• Efficient for local queries on small to medium datasets.
  
• Embeddable usage is a big advantage for data scientists or local prototyping.

Drawbacks:

• Not built for distributed processing or concurrency at scale.
  
• Community-driven integration is still evolving.

8. StarRocks

StarRocks is an open-source, distributed MPP database for high-performance analytics on large-scale datasets. It is known for its high concurrency, low latency, and ability to handle complex queries efficiently. StarRocks is often used for real-time analytics, ad-hoc queries, and data warehousing.

Integration Approach:

• StarRocks provides an Iceberg connector that can map Iceberg tables to StarRocks external tables.
  
• You can also import data from Iceberg directly into StarRocks’ internal storage.

Read/Write Capabilities:

• Supports batch reading, partial real-time ingestion, and near real-time updates.
  
• Example:

plaintext
CREATE EXTERNAL TABLE iceberg_orders (
    order_id INT,
    product_id INT,
    order_date DATETIME
)
ENGINE=ICEBERG
PROPERTIES (
    "iceberg.catalog.type" = "hive",
    "iceberg.catalog.hive.metastore.uris" = "thrift://hive-metastore:9083",
    "iceberg.db" = "db",
    "iceberg.table" = "orders"
);

Metadata Handling:

• StarRocks recognizes Iceberg snapshots and partition schemes.
  
• Some advanced Iceberg features like schema evolution are supported, but version compatibility matters.

Performance Characteristics:

• Optimized for high-concurrency, real-time analytics scenarios.
  
• Columnar storage and vectorized execution provide low-latency queries.

Drawbacks:

• Iceberg integration is relatively new, so new releases may introduce breaking changes.
  
• Production readiness depends on the alignment of the StarRocks version with the Iceberg releases.

9. ClickHouse

ClickHouse is an open-source, column-oriented database management system for real-time analytics and data warehousing. It is known for its high performance, scalability, and ability to efficiently handle large volumes of data. ClickHouse is often used for business intelligence, fraud detection, and log analysis.

Integration Approach:

• There is community-driven support for reading Iceberg data in ClickHouse.
  
• Primarily done through external table definitions or custom table engines.

Read/Write Capabilities:

• Batch reads for OLAP queries.
  
• Write support is limited or non-existent in stable releases.
  
• Example usage (hypothetical):

plaintext
CREATE TABLE iceberg_table 
ENGINE = Iceberg(...)
AS SELECT * FROM some_other_table;

Metadata Handling:

• Limited snapshot and partition management.
  
• Community plugins may require manual refresh of metadata.

Performance Characteristics:

• ClickHouse excels at real-time analytics when data is in its native format.
  
• Query performance for external data depends on connector maturity.

Drawbacks:

• Iceberg support is still experimental.
  
• Concurrency and metadata synchronization remain challenging in some cases.

10. Google BigQuery

BigQuery is a serverless, highly scalable, and cost-effective cloud data warehouse. It is a fully managed service that enables businesses to analyze large amounts of data quickly and easily. BigQuery is often used for business intelligence, data mining, and machine learning applications.

Integration Approach:

• BigQuery can query Iceberg tables through BigLake tables or external tables.
  
• Data is typically stored on GCS, with BigQuery referencing it externally.

Read/Write Capabilities:

• Primarily read queries with external or BigLake tables.
  
• Writes require either federated CTAS or loading data into BigQuery’s native tables first.
  
• Example:

plaintext
CREATE EXTERNAL TABLE dataset.iceberg_table
WITH CONNECTION `connection_id`
OPTIONS (
  format = 'PARQUET',
  uris = ['gs://bucket/iceberg/db/table/part-*.parquet']
);

SELECT * FROM dataset.iceberg_table;

Metadata Handling:

• BigQuery does not directly manage Iceberg snapshots.
  
• You must refresh the external table or rely on partition auto-discovery.

Performance Characteristics:

• Suitable for on-demand queries if you prefer serverless pricing.
  
• Performance depends on external data location and partition pruning.

Drawbacks:

• Minimal direct write capability to Iceberg.
  
• Some Iceberg features (time travel) are unavailable in external tables.

Choosing the Right Query Engine for Your Iceberg Workloads

When evaluating which query engine best suits your Apache Iceberg–based environment, it’s helpful to frame the decision around business objectives, technical constraints, and future scalability. Below are key considerations to help you select the most appropriate engine for your workloads without delving into specific code or configurations:

1. Streaming vs. Batch Requirements
   • High-Frequency Updates or Real-Time Pipelines: If you need continuous ingestion and near real-time insights (e.g., CDC pipelines, event streaming), engines with strong native streaming support like Apache Flink generally stand out.
   • Periodic or On-Demand Jobs: Apache Spark is a common choice for batch-oriented processing, such as scheduled ETL, large-scale transformations, or micro-batch workloads. It balances extensive ecosystem support with robust performance optimizations.
     
2. Interactive Analytics and Concurrency
   • Ad-Hoc Querying and Federated Analysis: If your organization relies on fast SQL access across diverse data sources and needs to handle concurrent user sessions (e.g., data science teams, BI analysts), Trino or Dremio can offer low-latency execution and a seamless user experience.
   • Real-Time OLAP and High Concurrency: For workloads demanding sub-second response times and many simultaneous queries, engines like StarRocks or a carefully tuned ClickHouse environment may provide the necessary concurrency and performance, significantly when your data structure can benefit from columnar optimizations.
     
3. Ecosystem Integration and Operational Simplicity
   • Cloud-Native Architectures: If you operate primarily in a managed data warehouse environment, Snowflake or BigQuery external tables can help you read from Iceberg with minimal operational overhead. This is often preferred when your team wants to centralize analytics within an existing cloud data platform without adopting new infrastructure.
   • Hadoop Heritage and Legacy Environments: Apache Hive remains a straightforward option if you’re already running a Hadoop-centric stack and want a gradual migration path. As part of a long-term architecture refresh, it can serve as a stepping stone to more modern engines.
     
4. Metadata and Governance Needs
   • All engines listed support Iceberg’s transactional metadata features at varying levels, including snapshot reads and partition evolution. Ensure your chosen engine has “full” (or at least partial) metadata support that is aligned with your governance strategy. If your workloads involve frequent schema changes or advanced time-travel queries, confirm that these features are well-tested in your target engine.
     
5. Resource Management and Scalability
   • Cluster Tuning and Cost Considerations: Spark and Flink typically require more hands-on tuning around shuffle operations, checkpoint intervals, and cluster sizing. By contrast, engines like Snowflake or BigQuery offload more complexity to a fully managed service model, albeit at a different cost structure.
   • Memory Footprint and Query Optimization: Interactive engines (e.g., Dremio, Trino) often thrive on ample memory to handle large joins and complex queries. Plan for capacity and data partitioning strategies to avoid performance bottlenecks.
     
6. Maturity and Community Support
   • For some engines, Iceberg integration is relatively new or experimental (e.g., ClickHouse, DuckDB). This can be acceptable for proof-of-concepts, local analytics, or smaller data volumes. For enterprise-critical deployments, you may need robust community or commercial support, frequent releases, and a proven track record of handling production-scale datasets.
     
7. Future-Proofing
   • Evolving Requirements: Organizations often start with batch workloads and gradually expand toward real-time or advanced analytics use cases. Selecting an engine with a clear roadmap for enhancements around streaming, interactive queries, or ML workflows can protect your investment over time.
   • Vendor and Ecosystem Alignment: Confirm that your chosen engine can adapt if you anticipate future cloud migrations or a transition to microservices-based data processing. This includes multi-cloud or hybrid-cloud support, flexible catalog integrations, and straightforward paths to scale up or down as data volumes change.

Real-World Use Cases of Iceberg Query Engines

To illustrate how these query engines perform in real-world scenarios, here are two practical examples showcasing common use cases: real-time CDC pipelines and interactive BI analytics.

Example: Real-Time CDC Pipeline

Let’s take a look at a common practical scenario. You must continuously capture changes from a MySQL database and apply them to an Apache Iceberg table so downstream systems can consume fresh data in near real-time. A common approach is to pair Estuary Flow with Apache Flink:

1. Capture Changes with Estuary Flow
   • Configure Estuary Flow to read MySQL binlogs, which track all inserts, updates, and deletes. Estuary offers a no-code MySQL connector so you can do this in a few minutes.
   • Forward these changes to an Iceberg table using Estuary Flow’s Iceberg connector, ensuring each captured event lands in your data lake storage.
     
2. Stream Processing via Apache Flink
   • Point a Flink job at the newly populated Iceberg table.
   • Perform real-time transformations or enrichments on incoming data and optionally write back to another Iceberg table for additional processing or analytics.
     
3. End-to-End Operations
   • Run this pipeline continuously so each new MySQL change is reflected in Iceberg with minimal delay.
   • Use Iceberg’s snapshot isolation to maintain consistent views and avoid conflicts during writes.

Key Considerations

• Checkpointing and Fault Tolerance: Configure Flink checkpoints to handle restarts gracefully, preserving state when failures occur.
  
• Metadata Store: Use a reliable external catalog (e.g., Hive Metastore, AWS Glue) to track Iceberg table metadata.
  
• Partition Evolution: If data distribution changes over time, leverage Iceberg’s partition evolution capabilities to optimize query performance.
  
• Compaction and File Management: Implement compaction strategies to merge small files, avoiding read overhead and ensuring efficient queries.

Example: Interactive Analytics and BI

As a second example, let's see how you could build a lakehouse stack for BI. You want to empower your organization’s analysts and data scientists to query diverse data sources in real-time for ad-hoc analytics, dashboard creation, or exploratory data mining. By combining Estuary Flow with engines like Trino or Dremio, you can unify data ingestion, transformation, and interactive querying under a single architecture:

1. Data Ingestion via Estuary Flow
   • Configure Estuary Flow to capture and merge data from multiple sources (e.g., relational databases, event streams, APIs) into Apache Iceberg tables.
   • Leverage Estuary Flow's streaming or batch connectors to ensure each source lands in a consistent format and schema within your Iceberg data lake.
     
2. Install and Configure the Iceberg Connector
   • Enable the Iceberg connector in Trino or Dremio so these engines can discover and work with your Iceberg tables.
   • Point the connector to the metadata catalog (e.g., Hive Metastore, AWS Glue) used by Estuary Flow and your Iceberg environment.
     
3. Unified Interactive Queries
   • Present Iceberg datasets as logical schemas accessible to analysts through JDBC/ODBC or native SQL interfaces.
   • Allow users to combine data from Iceberg tables with other sources (relational DBs, CSV files, or object stores), all in a single federated query engine.
     
4. Optimized Analytics
   • Encourage row-level filtering and partition pruning to minimize scanned data and optimize performance.
   • Apply query accelerations (e.g., Dremio Reflections) or caching features to reduce response times for repetitive queries.

Key Considerations

• Compaction and Metadata Refresh: Schedule regular compaction tasks to handle small file consolidation in Iceberg, preserving query performance for interactive workloads.
  
• Table Layout and Partition Strategy: Continuously refine partition columns and data distribution based on query patterns to further reduce scan costs.
  
• Resource Management: Monitor concurrency and memory allocations, especially for large joins or multiple simultaneous queries, to maintain low-latency performance.
  
• Governance and Security: Integrate with enterprise authentication and authorization systems so only authorized users can query data ingested by Estuary Flow. This ensures consistent governance across all data sources in your organization.

Loading Data Into Apache Iceberg

Query engines can be powerful tools for analyzing data, but they are only helpful if the data is already in the Iceberg format. This can be a significant limitation, as many organizations store data in various formats, including legacy systems, data warehouses, and cloud storage.

Estuary Flow fills this gap by providing a unified data movement platform integrating streaming and batch data sources into Iceberg. It is also highly scalable and can handle large volumes of data.

To learn more about how Estuary Flow integrated with Iceberg, check out these articles:

1. Getting Started With Iceberg: The Ultimate Guide
2. How to load data into Iceberg
3. Catalog Showdown: Polaris vs Unity
4. Best Tools to Load Data Into Iceberg

Conclusion

Apache Iceberg offers a robust way to handle evolving schemas, snapshots, and large-scale analytic datasets. The choice of query engine depends heavily on your workload (stream vs. batch), ecosystem preferences, and performance requirements.

• Spark and Flink lead for large-scale and real-time pipelines.
  
• Trino, Dremio, and StarRocks excel at interactive analytics.
  
• Motherduck for a modern lakehouse-based SaaS analytics engine.
  
• Snowflake and BigQuery allow external reads with minimal overhead if you are in their ecosystems.
  
• DuckDB and ClickHouse have promising but evolving Iceberg support.

Each engine has its integration points, feature sets, and performance trade-offs. Use the summary table and detailed feature breakdown to guide your decision. Always test in a proof-of-concept environment with real data volumes and access patterns before committing to a final production architecture.