Self-serve streaming data platform for building real-time ETL from DB, SaaS and filestores. Company behind Gazette and Estuary Flow OSS.
Self serve tool for creating a Kafka cluster and using it for real-time ETL from DB's and Filestores. Company behind Apache Kafka.
n/a
Price
Open-Source, or predictably priced pipelines at $1/ GB plus $0.14 / hr (~$100/mo) for any capture or materialization.
Open Core and predictably priced based on several metrics. Initially you'll need a kafka cluster and then will be billed on connectors that you use, data transferred, and data stored.
Estuary's pricing saves 70% or more depending on your data scale.
Latency
<100ms. Only constraint is frequency of updates from the source, or what the destination can handle.
<100ms. Only constraint is frequency of updates from the source, or what the destination can handle.
N/A
Connectors
100+ connectors. Also HTTP file, webhook, and ability to spin up most new connectors within a week.
60+ connectors with half built in-house and half open-source. Materialization connectors use at-least once semantics vs exactly once and no capability for data reduction.
Extracting data from more of your systems can enhance outcomes. Exact semantics helps ensure data accuracy.
On-Prem
Coming Winter 2023
Yes
Confluent is a good solution for on-prem data flows.
Delivery
Exactly-Once
At-least Once
At-least once semantics can create duplicates the destination, creating inaccurate results and excess cost.
Schema Migrations
Automated Schema Evolution.
Users manage the Schema Registry to validate and evolve data and schema.
Automation ensures that your destination always matches your source.
Data is stored in the Kafka Kafka topic at ($0.10/gb/mo). This leads to significant costs and results in many users pushing data to a batch system.
By storing data in a real-time data lake, you can endlessly distribute in real-time off one ingest, saving you egress fees, money, and source system stress.
Transforms
Streaming SQL and javascript transforms with joins on both real-time and historical data. DBT as a destination.
Single-Message Transforms can perform basic transforms of a single message or can use ksql for broader streaming joins. Soon to be Flink with the Immerock acquisition.
Estuary is building the next generation of real-time data integration solutions.
We're creating a new kind of DataOps platform thatempowers data teams to build real-time,data-intensive pipelines and applications, at scale,with minimal friction, in a UI or CLI. We aim to make real-time data accessible to the analyst, while bringing power tooling to the streaming enthusiast. Flow unifies a team's databases, pub/sub systems, and SaaS around their data, without requiring new investments in infrastructure or development.
Estuary develops in the open to produce both the runtime for our managed service and an ecosystem ofopen-source connectors. You can read more about our story here.
