Estuary

The Estuary runtime

Discover the underlying tech behind Estuary's best-in-class data movement.

  • 200GBper hour, per worker
  • Scaleat any time
One transactionSale reversalInventory updateLoyalty adjustmentcommittedDestinationSale reversalInventory updateLoyalty adjustmentDownstream sees the whole transaction, or none of it.One transactionSale reversalInventory updateLoyalty adjustmentcommittedDestinationSale reversalInventory updateLoyalty adjustmentDownstream sees the whole transaction, or none of it.

The engine underneath every pipeline

You describe the pipeline. The runtime handles the rest.

Most streaming systems fix parallelism at write time. Partitions get decided when data lands, and each one maps to exactly one consumer. That works until you need more capacity, at which point a layout chosen months ago becomes the ceiling.

Estuary's runtime decides grouping at read time instead. Storage layout and compute scale independently, so a task adds shards without repartitioning anything upstream.

Durable log at the center

Captures write into collections. Collections are a durable ordered log in object storage. Materializations read from that log.

What happens between the log and the workers reading it is the secret sauce.

SOURCEPostgresCaptureCollectiondurable, ordered log in object storagestorage layout stays fixedMaterializationgrouping decided at read timeShard 1Shard 2Shard 3+ Shard 4add shards without repartitioningDESTINATIONSnowflakePostgresCaptureCollectiondurable, ordered log in object storageMaterializationShard 1Shard 2Shard 3+ S4Snowflake

Compute scales on its own

Add shards without touching the storage layout. Capacity is not a decision you make once, at the start, forever.

Reads and writes are independent

A dead destination doesn't stop the capture. The materialization resumes where it left off.

Backfill and streaming are one path

Same log, same order. No batch job to reconcile, no window where the two disagree.

  • Up to 8x faster backfills

  • Even faster on destinations that parallelize

  • Maintains transactional consistency

  • Clear observability on backfill progress

  • Up to 200 GB per hour per worker

Rewriting the runtime

Sometimes the best-laid plans are iterative. We've tuned up the Estuary runtime to bring you the fastest data possible at any scale.

Read the runtime whitepaper

Learn about the tech behind the runtime and the story behind the rewrites. Estuary:

  • Scales data movement to handle high throughputs

  • Ensures end-to-end transactional consistency

  • Decouples storage layout from shard grouping

The new Estuary runtime whitepaper

Trusted by data leaders

  • Together AI avatar

    YuTong (Julia) Zhang

    Senior Software Engineer, Together AI
    Together AI avatar

    For AI systems like ours, freshness of data is everything. Estuary gives us sub-second latency without the complexity of maintaining streaming infrastructure ourselves. That reliability means our teams can focus on advancing AI models instead of pipelines.

  • Glossier avatar

    Brandon Besash

    Director, Business Intelligence, Glossier
    Glossier avatar

    Estuary enabled us to finally implement our ERP’s new data endpoint with all our inventory transactions, purchasing, and shipping data. We can now unlock data blocked by cost before, and sync times are much faster and are always being improved by the Estuary team.

    Read the Success Story
  • Xometry avatar

    Andrew Woelfel

    Senior Manager, Data Engineering and Analytics, Xometry
    Xometry avatar

    “Estuary has been a pleasure to work with and has significantly modernized our data infrastructure, delivering real-time and scalable processes that will significantly impact company-wide operations. Every data-driven organization should be looking at Estuary today.”

    Read the Success Story
  • Cosuno avatar

    Maximilian Seifert

    CTO, Cosuno
    Cosuno avatar

    Estuary just works. We’ve never had an incident, and it cut our data movement costs in half.

    Read the Success Story
  • Shippit avatar

    Keat Min Woo


    We didn’t want to be locked into a system where faster syncs meant higher bills. Estuary gives us real-time pipelines without pricing games or the burden of running Kafka ourselves.

    Read the Success Story
  • Livble avatar

    Uri Vinetz

    Director of Data, Livble
    Livble avatar

    We needed something self-serve, fast, and reliable, and Estuary delivered exactly that. It’s a huge unlock for our operations, reporting, and machine learning.

    Read the Success Story
  • Resend avatar

    Jonni Lundy

    COO, Resend
    Resend avatar

    Estuary transformed how we operationalize our data for fraud, security, support, and beyond. Instead of unreliable, expensive backfills, we have real-time visibility into platform activity. The proactive support and hands-on approach make all the difference.

    Read the Success Story
  • Recart avatar

    Istvan Kovacs

    CTO, Recart
    Recart avatar

    Estuary became our real-time data backbone without the cost or complexity of traditional solutions. We replaced a fragile, high-maintenance pipeline with a managed system that just works and scales.

    Read the Success Story
  • Headset avatar

    Scott Vickers

    CTO, Headset
    Headset avatar

    Estuary has been a game-changer for Headset’s data infrastructure. Compared to our previous solutions, it has dramatically improved reliability while reducing our overall costs significantly.

    Read the Success Story
  • Revunit avatar

    Revunit


    Estuary is our preferred CDC solution for importing data from application databases into BigQuery for analytics. It offers a transparent pricing structure, timely support responses, and an intuitive CLI tool for bulk configuration tasks. In contrast, other market solutions often have ambiguous pricing and fewer options for precise data replication across environments. This makes choosing to use Estuary an obvious decision.

  • PDI. avatar

    PDI.


    Estuary makes tough data transformation problems a piece of cake with its intuitive user interface and incredible breadth of features.

  • OneCommerce avatar

    OneCommerce


    Estuary is the only SaaS tool that we found which can do a simple loop and calculate COGS from an array of objects nested in a property. We love to write transformations in typescript because it's in the same codebase and super easy to maintain and read. It's a true game changer.

  • Minima Global avatar

    Minima Global


    Getting #MINIMA real-time data replication out to the Postgres database was not fun until we found @EstuaryDev it is the best materialization.

  • Seattle Data Guy avatar

    Ben Rogojan

    Owner, Seattle Data Guy
    Seattle Data Guy avatar

    Estuary makes working with real-time data more cost effective and just as simple as working with batch data.

  • Pompato avatar

    Pompato


    This tool is 1000x times better than LogStash or Elastic Enterprise Data Ingestion Tool.

  • DeepSync avatar

    DeepSync


    Estuary allows us to integrate low-latency CDC and connect to SaaS apps across our entire reporting stack and it’s the only solution that we’ve found that lets us do both.

  • Fenestra avatar

    Fenestra


    We needed a platform to help us optimize marketing campaigns with low-latency. Estuary provided an unparalleled solution to do that at terabyte scale.

  • Coalesce avatar

    Coalesce


    Estuary is the only system we’ve found that can seamlessly replicate large scale Firestore data for analytics. After months of research and trying everything, we can confidently say that Estuary is the only company that can help us get easy, accurate analytics on our data within Snowflake when replicating from Firestore data.

  • Flashpack avatar

    Flashpack


    We're a big fan of Estuary's real-time, no code model. It's magic that we're getting real time data without much effort and we don't have to spend time thinking about broken pipelines. We've also experienced fantastic support by Estuary.