Streaming Data Lakehouse Tutorial: MongoDB to Apache Iceberg
Learn how to connect MongoDB to Apache Iceberg in Iceberg table format using Estuary. In this step-by-step demo, we show you how to:
1. Set up a MongoDB source and configure secure connections.
2. Create real-time pipelines to load data into Amazon S3.
3. Leverage the AWS S3 Iceberg Connector with AWS Glue for table cataloging.
Estuary simplifies real-time data integration with powerful features like advanced security connections, automated materialization, and streamlined pipeline management. Whether you're handling transactional data or syncing complex data streams, Estuary has you covered.
👉 Try Estuary: https://dashboard.estuary.dev/register
👉 Read the Documentation: https://docs.estuary.dev/
#MongoDBtoIceberg
0:00 - Introduction: Overview of the demo and Estuary.
0:07 - Step 1: Setting Up MongoDB Source: Configuring MongoDB as the data source.
0:44 - Step 2: Reviewing Collections: Selecting collections to sync.
1:03 - Step 3: Setting Up S3 Destination: Configuring the AWS S3 Iceberg connector.
1:37 - Step 4: Testing and Publishing Pipeline: Testing the connection and publishing the pipeline.
2:07 - Final Verification: Verifying MongoDB data in S3 as Iceberg tables.
More videos
PostgreSQL to Iceberg - Streaming Lakehouse Foundations
Stream Real-Time Data from Postgres to Iceberg with Change Data Capture and Estuary. In this step-by-step tutorial, we demonstrate how to set up and stream real-time data from a PostgreSQL database into Iceberg tables using change data capture (CDC) with Estuary. Learn how to capture, ingest, and materialize data using Estuary's seamless integration. This demo uses a sales database to showcase how changes in a PostgreSQL table are tracked and replicated into an Iceberg table stored in AWS S3. Check out Estuary's Iceberg integration: https://estuary.dev/destination/s3-iceberg/ Join Estuary's community Slack: https://estuary-dev.slack.com/join/shared_invite/zt-86nal6yr-VPbv~YfZE9Q~6Zl~gmZdFQ#/shared-invite/email 00:00 - Introduction: Streaming Data from Postgres to Iceberg 00:18 - Postgres Sales Database Overview 01:08 - Starting Change Data Capture (CDC) with Estuary 02:09 - Materializing Data into Apache Iceberg 04:17 - Backfilling Data into Iceberg 05:21 - Querying Iceberg Tables with Python 06:10 - Conclusion: Demo Recap
Real-time CDC with MongoDB and Estuary in 3 minutes
Build a Real-Time CDC Pipeline from MongoDB using Estuary: This tutorial demonstrates how to create a real-time change data capture (CDC) pipeline from MongoDB using Estuary. It covers setting up MongoDB Atlas, configuring Estuary, and monitoring data replication in real-time. #MongodbCDC #Changedatacapture Start building for free at: https://dashboard.estuary.dev/register Blog Post MongoDB CDC: https://estuary.dev/mongodb-change-data-capture/ 0:00 – Introduction: Real-Time CDC Pipeline from MongoDB using Estuary 0:07 – Provisioning MongoDB Atlas 0:56 – Creating a Real-Time CDC Pipeline in Estuary 1:17 – Discovering Database Objects for Replication 1:47 – Saving and Publishing the CDC Pipeline 2:36 – Inserting a New Record in MongoDB 3:01 – Verifying Record Update in Estuary
What’s Next for Data Warehouses? Lessons from Our Benchmark and Emerging Trends
Dani and Ben talks about key findings on performance ceilings, cost traps, and failure modes, and explore the major trends reshaping data warehouse architecture, including: - Separation of Compute & Storage – How Snowflake Gen2, Databricks serverless, and open table formats like Iceberg are changing the game. - Lakehouse Reality Check: What’s working for teams adopting Iceberg, schema evolution patterns, and lake-native pipelines. - Flexibility Over Centralization: Moving beyond “one warehouse to rule them all.
How to Connect NetSuite to Estuary for Real-Time Data Integration
Learn how to connect NetSuite to Estuary and stream your ERP data in real-time to destinations like Snowflake, BigQuery, Databricks, Redshift, and more — no custom ETL code required. This tutorial walks through the full NetSuite setup (SuiteAnalytics, integration, role, user, and token-based authentication) and the Estuary capture configuration end-to-end. 👉 Try Estuary Free: https://dashboard.estuary.dev/register 👉 Book a Demo: https://estuary.dev/demo 👉 NetSuite Connector Docs: https://docs.estuary.dev/reference/Connectors/capture-connectors/netsuite/ ━━━━━━━━━━━━━━━━━━━━━━━ WHAT YOU'LL LEARN ━━━━━━━━━━━━━━━━━━━━━━━ ✅ How to enable SuiteAnalytics in NetSuite ✅ How to create a NetSuite integration with token-based authentication ✅ How to set up a role and assign permissions for Estuary access ✅ How to generate NetSuite access tokens (consumer key, secret, token ID, token secret) ✅ How to configure the Estuary NetSuite capture connector ✅ How to find your NetSuite account ID and internal role ID ━━━━━━━━━━━━━━━━━━━━━━━ CHAPTERS ━━━━━━━━━━━━━━━━━━━━━━━ 0:00 Intro: NetSuite to Estuary Integration 0:18 NetSuite Setup: SuiteAnalytics, Account ID, Integration 2:08 Create Role and Assign User Permissions 2:37 Generate NetSuite Access Tokens 3:25 Enable Internal IDs in NetSuite 4:02 Create the NetSuite Capture in Estuary 5:16 Endpoint Configuration and Authentication 5:43 Test, Save, and Publish the Capture ━━━━━━━━━━━━━━━━━━━━━━━ WHY NETSUITE WITH ESTUARY ━━━━━━━━━━━━━━━━━━━━━━━ NetSuite is one of the most widely used ERPs for finance, operations, and inventory data, but getting that data into a modern warehouse is notoriously painful. Estuary is a real-time data platform that streams NetSuite data into analytics destinations with sub-second latency — without requiring custom SuiteScript, scheduled CSV exports, or homegrown ETL pipelines. Common NetSuite pipelines built on Estuary: - NetSuite to Snowflake - NetSuite to BigQuery - NetSuite to Databricks - NetSuite to Redshift - NetSuite to PostgreSQL ━━━━━━━━━━━━━━━━━━━━━━━ CONNECT WITH ESTUARY ━━━━━━━━━━━━━━━━━━━━━━━ 🌐 Website: https://estuary.dev/ 💬 Slack Community: https://estuary-dev.slack.com/ 📚 Docs: https://docs.estuary.dev #NetSuite #DataIntegration #ERP #DataPipeline #Estuary #SuiteAnalytics #NetSuiteToSnowflake #DataEngineering #RealTimeData #ETL
Stream Data to Apache Iceberg with Estuary
Learn about the Apache Iceberg table format, why it’s essential for organizing your data lake, and how to load data into Iceberg using Estuary. We’ll cover a brief intro to Iceberg before demoing the connector setup with Estuary, Amazon S3, and AWS Glue for real-time and batch data integration. With Estuary, you can stream structured or unstructured data directly into Iceberg tables — whether your source is PostgreSQL, Kafka, Snowflake, MongoDB, or many others — making it easy to build a scalable, query-ready data lakehouse architecture. Find more at Estuary’s: - Website: https://estuary.dev/ - Docs: https://docs.estuary.dev/ - Introduction to Iceberg: https://estuary.dev/apache-iceberg-tutorial-guide/ - Iceberg connector documentation: https://docs.estuary.dev/reference/Connectors/materialization-connectors/amazon-s3-iceberg/ #ApacheIceberg #datalakehouse Media resources used in this video are from Pexels and the YouTube Studio Audio Library. 0:00 Intro 1:00 What is Iceberg? 2:30 Beginning connector setup in Estuary 3:17 AWS resources 5:00 Additional config and catalogs 6:05 Wrapping up connector creation 6:36 Review and outro
MongoDB to Snowflake in real-time (no Debezium)
In this video, Jeff from Estuary walks you through how to move data from MongoDB to Snowflake using Estuary, a real-time ETL platform. Learn the key benefits of using Estuary, including low-latency Change Data Capture (CDC) and automatic unpacking of nested documents. You'll also see a step-by-step guide to setting up a MongoDB Atlas database and creating a real-time data pipeline with Estuary. Key features covered: - Real-time data replication from MongoDB to Snowflake - Low-latency data movement and automatic flattening of nested documents - Backfilling data and setting up materializations to Snowflake in just a few clicks #MongodbtoSnowflake #changedatacapture If you have any questions, feel free to join our community Slack. Start building real-time data pipelines with Estuary today! Sign up for a free account: https://dashboard.estuary.dev/register Join our Slack community: https://estuary-dev.slack.com/join/shared_invite/zt-86nal6yr-VPbv~YfZE9Q~6Zl~gmZdFQ#/shared-invite/email Blog: https://estuary.dev/mongodb-to-snowflake/ 0:00 – Introduction: Moving Data to Snowflake with Estuary 0:12 – Key Benefits of Using Estuary: Real-Time Data Integration 1:18 – Automatic Flattening of Nested Data 2:08 – Testing Connection to the MongoDB Source 2:25 – Saving and Publishing the Real-Time Pipeline 2:54 – Sending Data to Snowflake and Other Destinations 3:12 – Real-Time Backfill and Data Materialization to Snowflake
Estuary Overview
Discover the power of Estuary, a platform built to make creating real-time data pipelines easy. In this overview, we’ll show you how Estuary helps you move data from source to destination in real time, with no coding required. 🌐 Check out our website to learn more about Estuary: https://www.estuary.dev/ ➡️ Start building your pipelines for free now: https://dashboard.estuary.dev/register if you’re curious for more, check out our docs or jump into our community Slack to ask questions! 📚 Explore our docs for detailed guides and tutorials: https://docs.estuary.dev/ 💬 Join our Slack community to connect with developers and ask questions: https://estuary-dev.slack.com/ #Estuary #RealtimeETL #DataStreaming #DataOps #dataengineering
Seamless Data Integration, Unlimited Potential
Discover the simplest way to connect and move your data.Get hands-on for free, or schedule a demo to see the possibilities for your team.