Amazon RDS to Snowflake in Real Time (Postgres CDC)
Move data from Amazon RDS (PostgreSQL) to Snowflake in real time using change data capture, with no batch jobs and no custom code. In this short demo, Estuary captures live changes from an RDS Postgres database and materializes them into Snowflake in minutes.
How do you sync RDS Postgres to Snowflake in real time? Estuary captures inserts, updates, and deletes from your Postgres database, lands them in a collection, and materializes that collection into Snowflake with low latency. Insert a row in Postgres and watch it appear in Snowflake seconds later, no pipeline to build or maintain.
⏱️ Chapters
0:00 Real-time data integration with Estuary
0:11 Setting up Amazon RDS PostgreSQL as a source
0:27 Configuring the Estuary source connector
0:51 Testing and publishing the source
1:09 Creating a collection for real-time data
1:36 Materializing data to Snowflake
2:01 Testing real-time data ingestion
🔑 What you'll learn
- How real-time CDC differs from batch ETL for loading Snowflake
- Connecting an Amazon RDS PostgreSQL source in Estuary
- Creating a collection and materializing it to Snowflake
- Verifying the live sync by inserting a record in Postgres
🔗 Try Estuary free: https://dashboard.estuary.dev/register
📚 Docs: https://docs.estuary.dev
📝 Tutorial: https://estuary.dev/blog/tutorial/
🛠️ Tools: Amazon RDS, PostgreSQL, Snowflake, Estuary
#Snowflake #Postgres #CDC #ChangeDataCapture #DataEngineering #RealTimeData #AmazonRDS #DataPipeline #ETL #Estuary
More videos
PostgreSQL CDC Setup Tutorial: Real-Time Data Capture with Estuary (Step-by-Step)
Learn how to set up PostgreSQL Change Data Capture (CDC) with Estuary in under 10 minutes. This tutorial walks through enabling logical replication on Google Cloud SQL, creating the replication user, and connecting your Postgres database to Estuary for real-time streaming to Snowflake, BigQuery, Databricks, and more. ⚠️ Heads up: This video was recorded on an earlier version of Estuary. The core concepts (logical replication, replication user, Cloud SQL flags) still apply, but the UI and a few setup steps have been updated since. For the latest instructions, follow the docs: https://docs.estuary.dev/reference/Connectors/capture-connectors/PostgreSQL/ 👉 Try Estuary Free: https://dashboard.estuary.dev/register 👉 Book a Demo: https://estuary.dev/demo 👉 Full Docs: https://docs.estuary.dev/reference/Connectors/capture-connectors/PostgreSQL/ ━━━━━━━━━━━━━━━━━━━━━━━ WHAT YOU'LL LEARN ━━━━━━━━━━━━━━━━━━━━━━━ ✅ How to enable logical replication on Cloud SQL PostgreSQL ✅ How to create a dedicated replication user with correct privileges ✅ How to configure the Estuary PostgreSQL capture connector ✅ How to troubleshoot connection string and IP allowlist errors ━━━━━━━━━━━━━━━━━━━━━━━ CHAPTERS ━━━━━━━━━━━━━━━━━━━━━━━ 0:00 Intro: PostgreSQL CDC with Estuary 0:52 Enable Logical Replication on Cloud SQL PostgreSQL 2:45 Create Estuary Capture for PostgreSQL 3:00 PostgreSQL Endpoint Configuration (Host, Port, User) 3:55 Troubleshooting: No Collections Found in Source Database 5:14 Troubleshooting: PostgreSQL Connection String Errors ━━━━━━━━━━━━━━━━━━━━━━━ WHY POSTGRESQL CDC WITH ESTUARY ━━━━━━━━━━━━━━━━━━━━━━━ Estuary is a real-time data platform built for Change Data Capture. Unlike batch ETL tools, Estuary streams row-level changes from PostgreSQL's write-ahead log (WAL) to destinations like Snowflake, BigQuery, Databricks, Redshift, and ClickHouse with sub-second latency. Works with PostgreSQL on: - Google Cloud SQL - Amazon RDS / Aurora - Azure Database for PostgreSQL - Supabase, Neon, Heroku - Self-hosted PostgreSQL ━━━━━━━━━━━━━━━━━━━━━━━ CONNECT WITH ESTUARY ━━━━━━━━━━━━━━━━━━━━━━━ 🌐 Website: https://estuary.dev/ 💬 Slack Community: https://estuary-dev.slack.com/ 📚 Docs: https://docs.estuary.dev/ #PostgreSQL #CDC #ChangeDataCapture #DataEngineering #DataPipeline #RealTimeData #Estuary #CloudSQL #LogicalReplication #ETL
PostgreSQL CDC: What, Why, and How
Set up PostgreSQL change data capture (CDC) without worrying about your WAL or replication slot loss. This step-by-step Postgres CDC guide covers WAL configuration, replication user setup, publication creation, and building your first real-time pipeline to Snowflake, Iceberg, or ClickHouse, no Kafka or Debezium required. We walk through why CDC beats batch replication for Postgres, how the WAL and replication slots actually work (and what goes wrong), and a full setup demo with Estuary, a real-time CDC platform that manages WAL advancement and decouples your source from your destination. LINKS - Set up your first data pipeline for free at Estuary: https://dashboard.estuary.dev/register/?utm_source=youtube&utm_medium=social&utm_campaign=postgres - Discover more PostgreSQL configuration options in Estuary’s docs: https://docs.estuary.dev/reference/Connectors/capture-connectors/PostgreSQL/ - Explore available destinations to continue your pipeline: https://docs.estuary.dev/reference/Connectors/materialization-connectors/ - Join Estuary on Slack: https://go.estuary.dev/slack - Contact us: https://estuary.dev/contact-us FAQ 1. What is Postgres CDC? CDC, or change data capture, is a method to record all updates made to a database. In conjunction with PostgreSQL, CDC often uses the Postgres write-ahead log for data replication. We start discussing Postgres CDC at 3:36 2. When should you use CDC? Is it just for real-time data? CDC can be beneficial for many different use cases since it has a low impact on the replicated Postgres database and handles replication more efficiently than batch queries. Since CDC is also very low latency, it’s a top choice for real-time use cases, though batch workflows will still benefit from an efficiency/low-impact standpoint. We go over CDC use cases at 2:29 3. Is CDC complicated to implement? CDC can be complex when creating and managing a DIY solution. You need to be careful that the WAL doesn’t grow beyond expected bounds. A managed solution can make setup and maintenance much simpler; just make sure that the integration platform follows CDC best practices, like keeping the source and destination decoupled. A brief implementation guide with Estuary starts at 5:55 Media resources used in this video are from Canva and the YouTube Studio Audio Library. CHAPTERS 0:00 Intro 0:45 On PostgreSQL 2:05 Change Data Capture 3:36 Postgres + CDC 5:55 Demo: Postgres setup 7:36 Demo: Basic Estuary setup 8:52 Demo: Additional settings 10:35 Wrap up
Change Data Capture for PostgreSQL with Estuary
In this quick tutorial, Dani demonstrates how to effortlessly set up a PostgreSQL Change Data Capture (CDC) pipeline using Estuary in less than a minute. Watch as he works with a live sales table, showing you just how simple it is to connect your Postgres database and start replicating data in real time. You'll also see how Estuary handles schema capturing and backfills your existing data, making real-time data integration both fast and efficient. #PostgresCDC #changedatacapture Try Estuary today for real-time data pipelines and seamless integration with your databases! - Sign up for a free account: https://dashboard.estuary.dev/register - Join our Slack community: https://estuary-dev.slack.com/join/shared_invite/zt-86nal6yr-VPbv~YfZE9Q~6Zl~gmZdFQ#/shared-invite/email - Make sure to check out our Postgres CDC guide: https://estuary.dev/the-complete-change-data-capture-guide-for-postgresql/ Key things covered: 0:00 – Introduction: Setting up PostgreSQL CDC Pipeline 0:09 – Sales Table Example and Real-Time Updates 0:21 – Creating a Postgres Capture in Estuary 0:33 – Verifying the Sales Table and Schema 0:44 – Backfilling Data and Real-Time Replication
Estuary | The Right Time Data Platform
Welcome to Estuary, the Right Time Data Platform built for modern data teams. With Estuary, you can move and transform data between hundreds of systems at sub second latency or in batch, depending on your business needs. • Capture data from source systems using pre built, no code connectors. • Automatically infer schemas and manage both real time and historical events in collections. • Materialize your data to any destination with ease and flexibility. • Choose your deployment model: fully SaaS, Bring Your Own Cloud, or private deployment with enterprise level security. Start streaming an ocean of data and get going today: 🌊 https://dashboard.estuary.dev/register/?utm_source=youtube&utm_medium=social&utm_campaign=overview_video Learn more: 🌐 On our site: https://www.estuary.dev/?utm_source=youtube&utm_medium=social&utm_campaign=flow_overview 📚 In our docs: https://docs.estuary.dev/?utm_source=youtube&utm_medium=social&utm_campaign=flow_overview Connect with us: 💬 On Slack: https://go.estuary.dev/slack 🧑💻 In GitHub: https://github.com/estuary ℹ️ On LinkedIn: https://www.linkedin.com/company/estuary-tech/ #righttimedata #datapipelines #streamingdata #realtimeanalytics #CDC #dataengineering #Estuary
Real-time Data Products with Estuary - Ad Performance
In this detailed demo, Dani shows you how to build a real-time data product using Estuary. Learn how to capture and process data from a PostgreSQL database using Change Data Capture (CDC) and stream it into Snowflake for real-time ad performance calculations. This tutorial covers how to join and transform data using Estuary’s derivations and materialize the final output into Snowflake. Follow along as we work with ad clicks and impressions to perform transformations in real-time! 00:00 - Introduction: Real-Time Data Product with Postgres and Snowflake 01:30 - Setting Up Postgres CDC with Estuary 04:45 - Defining Derivations for Real-Time Transformations 09:45 - Materializing Data into Snowflake 12:03 - Real-Time Ad Performance Calculation in Snowflake 12:46 - Conclusion: Wrap-up and Further Resources
Introducing Estuary Agent Skills: Build Data Pipelines with AI
Estuary Agent Skills give your AI coding assistant the Estuary-specific context it needs to build and operate real-time data pipelines. Ask in plain English, capture my Postgres database into Snowflake, load these collections into BigQuery, or why is this connector failing, and your assistant knows the workflow. How do you build a data pipeline with an AI assistant? Instead of bouncing between docs, CLI commands, connector setup, and warehouse settings, Estuary Agent Skills follow the open SKILL.md pattern, so your assistant knows which flowctl commands to run, what specs to generate, and which checks to perform before publishing. You stay in control: you review the specs, approve the commands, and decide what credentials exist and what gets published. The skills are open source, so you can inspect everything. ⏱️ Chapters 0:00 The glue-work problem 0:29 Introducing Agent Skills 0:38 Pipelines in plain English 0:54 What's included 1:20 Demo: Postgres to Snowflake 1:44 You stay in control 2:34 Works with your AI tools 2:52 Get started 🔑 In the first release - Source skills: Postgres, MySQL, MongoDB, SQL Server - Materialization skills: Snowflake, BigQuery, Redshift, Databricks, Postgres - Operational skills: task health, catalog status, publication history, and more 🔗 Get the skills on GitHub: https://github.com/estuary/agent-skills 📖 Setup guide: https://docs.estuary.dev/guides/agent-skills/ 💡 Learn more: https://estuary.dev/capabilities/agent-skills/ 🚀 Start free: https://dashboard.estuary.dev/register #DataEngineering #AgentSkills #AI #Estuary #ChangeDataCapture #Snowflake #Postgres #ClaudeCode #Cursor #RealTimeData
![[WEBINAR] Estuary Agent Skills: From Setup to Success thumbnail](/static/0bfcc292be14e15168775e4eda941993/71d4d/maxresdefault.webp)
[WEBINAR] Estuary Agent Skills: From Setup to Success
In this webinar, Dani Palma (Developer Relations, Estuary) walks through the new Estuary Agent Skills product launch, from installation to building and debugging an Estuary pipeline. Agent Skills lets your AI assistant create and monitor production-grade Estuary pipelines in the tool your team already uses. You describe the pipeline using plain language prompts. The agent builds it. You review, approve, and run it. Agent Skills are compatible with Claude Code, Cursor, Codex, and 45+ other agents that support the open SKILL.md standard. We currently offer 37 skills that cover captures, materializations, pipeline operations, derivations, and schema management. Interesting in trying out Agent Skills? Here's how to get started: Agent Skills docs: https://docs.estuary.dev/guides/agent-skills/ Agent Skills overview: https://www.youtube.com/watch?v=lc71ILyd89g Agent Skills GitHub repo: https://github.com/estuary/agent-skills
Estuary 101: Right Time Data Integration for ClickHouse
Join Zulf Qureshi (Head of Sales Engineering @ Estuary) for a fast-paced walkthrough of how to design and ship right-time data pipelines for ClickHouse with Estuary. In this session, you’ll get: Context: What “right-time” really means, where Estuary fits among batch vs. streaming and managed vs. self-hosted options, and why unified ingestion reduces cost and complexity. Live End-to-End Demo: Connect CDC sources, apply declarative transformations, and materialize data simultaneously into a warehouse, analytical engines, and object storage—plus a look at observability, error recovery, and real-world scenarios like schema drift and backfills. Estuary helps organizations streamline their data integration for AI, analytics, and operational workflows without the constraints of a rigid architecture. As the right-time data platform, Estuary replaces fragmented CDC, streaming, and batch pipelines with one managed system and predictable pricing. Try Estuary for free today: https://dashboard.estuary.dev/register/
How to Connect NetSuite to Estuary for Real-Time Data Integration
Learn how to connect NetSuite to Estuary and stream your ERP data in real-time to destinations like Snowflake, BigQuery, Databricks, Redshift, and more — no custom ETL code required. This tutorial walks through the full NetSuite setup (SuiteAnalytics, integration, role, user, and token-based authentication) and the Estuary capture configuration end-to-end. 👉 Try Estuary Free: https://dashboard.estuary.dev/register 👉 Book a Demo: https://estuary.dev/demo 👉 NetSuite Connector Docs: https://docs.estuary.dev/reference/Connectors/capture-connectors/netsuite/ ━━━━━━━━━━━━━━━━━━━━━━━ WHAT YOU'LL LEARN ━━━━━━━━━━━━━━━━━━━━━━━ ✅ How to enable SuiteAnalytics in NetSuite ✅ How to create a NetSuite integration with token-based authentication ✅ How to set up a role and assign permissions for Estuary access ✅ How to generate NetSuite access tokens (consumer key, secret, token ID, token secret) ✅ How to configure the Estuary NetSuite capture connector ✅ How to find your NetSuite account ID and internal role ID ━━━━━━━━━━━━━━━━━━━━━━━ CHAPTERS ━━━━━━━━━━━━━━━━━━━━━━━ 0:00 Intro: NetSuite to Estuary Integration 0:18 NetSuite Setup: SuiteAnalytics, Account ID, Integration 2:08 Create Role and Assign User Permissions 2:37 Generate NetSuite Access Tokens 3:25 Enable Internal IDs in NetSuite 4:02 Create the NetSuite Capture in Estuary 5:16 Endpoint Configuration and Authentication 5:43 Test, Save, and Publish the Capture ━━━━━━━━━━━━━━━━━━━━━━━ WHY NETSUITE WITH ESTUARY ━━━━━━━━━━━━━━━━━━━━━━━ NetSuite is one of the most widely used ERPs for finance, operations, and inventory data, but getting that data into a modern warehouse is notoriously painful. Estuary is a real-time data platform that streams NetSuite data into analytics destinations with sub-second latency — without requiring custom SuiteScript, scheduled CSV exports, or homegrown ETL pipelines. Common NetSuite pipelines built on Estuary: - NetSuite to Snowflake - NetSuite to BigQuery - NetSuite to Databricks - NetSuite to Redshift - NetSuite to PostgreSQL ━━━━━━━━━━━━━━━━━━━━━━━ CONNECT WITH ESTUARY ━━━━━━━━━━━━━━━━━━━━━━━ 🌐 Website: https://estuary.dev/ 💬 Slack Community: https://estuary-dev.slack.com/ 📚 Docs: https://docs.estuary.dev #NetSuite #DataIntegration #ERP #DataPipeline #Estuary #SuiteAnalytics #NetSuiteToSnowflake #DataEngineering #RealTimeData #ETL
PostgreSQL to Iceberg - Streaming Lakehouse Foundations
Stream Real-Time Data from Postgres to Iceberg with Change Data Capture and Estuary. In this step-by-step tutorial, we demonstrate how to set up and stream real-time data from a PostgreSQL database into Iceberg tables using change data capture (CDC) with Estuary. Learn how to capture, ingest, and materialize data using Estuary's seamless integration. This demo uses a sales database to showcase how changes in a PostgreSQL table are tracked and replicated into an Iceberg table stored in AWS S3. Check out Estuary's Iceberg integration: https://estuary.dev/destination/s3-iceberg/ Join Estuary's community Slack: https://estuary-dev.slack.com/join/shared_invite/zt-86nal6yr-VPbv~YfZE9Q~6Zl~gmZdFQ#/shared-invite/email 00:00 - Introduction: Streaming Data from Postgres to Iceberg 00:18 - Postgres Sales Database Overview 01:08 - Starting Change Data Capture (CDC) with Estuary 02:09 - Materializing Data into Apache Iceberg 04:17 - Backfilling Data into Iceberg 05:21 - Querying Iceberg Tables with Python 06:10 - Conclusion: Demo Recap
Stream Real‑Time Data to Databricks with Estuary
Learn how to stream real-time data from PostgreSQL into Databricks using Estuary — no code, no maintenance. In this demo, Dani walks through: • Setting up a Databricks SQL Warehouse and generating a personal access token • Capturing the users and transactions tables from PostgreSQL in Estuary • Materializing those tables directly into Databricks using the built-in connector • Monitoring live data replication and verifying the results in Databricks SQL Warehouse Highlights of Estuary: • Real-time change data capture with millisecond latency • Native support for Databricks Unity Catalog and Delta Lake • Zero-code pipeline setup with automatic backfill and continuous sync Why it matters Streaming live data into Databricks unlocks fresh analytics, real‑time dashboards, and feeding ML models with up‑to‑date data — all without complex ETL or scripting 🔗 Learn more & get started: • Official Estuary guide: https://estuary.dev/real-time-fraud-detection-databricks/ • https://estuary.dev/blog/load-data-into-databricks/ • Start building for free at: https://dashboard.estuary.dev/register If you have questions or need help, jump into our community Slack or check the docs. #databrickstutorial #databricks 00:00 - Introduction 00:49 - Materializing Data to Databricks 01:45 - Verifying Data in Databricks 02:04 - Conclusion
Seamless Data Integration, Unlimited Potential
Discover the simplest way to connect and move your data.Get hands-on for free, or schedule a demo to see the possibilities for your team.