How to Set Up a Real-Time Shopify to BigQuery Integration

Syncing Shopify to BigQuery is one of the most effective ways to power advanced e-commerce analytics. Whether you're tracking orders in real time, analyzing customer behavior, or monitoring inventory across multiple locations, Google BigQuery gives you the speed and scale needed for meaningful insights.

However, building and maintaining a pipeline from Shopify to BigQuery isn’t trivial. Shopify’s GraphQL API has rate limits, bulk query constraints, and complex nested data. Traditional ETL tools often rely on batch updates, leading to delays and stale data.

Estuary Flow offers a modern alternative: a real-time data integration platform that captures data from Shopify and streams it directly to BigQuery. With native connectors and a streaming-first architecture, Flow eliminates the need for complex orchestration, custom scripts, or manual backfills.

In this guide, we’ll show you how to set up a Shopify BigQuery integration using Estuary — from capturing product and order data to materializing it into BigQuery tables, enabling fast, reliable, and real-time Shopify analytics.

Why Sync Shopify to BigQuery?

Shopify is a powerful e-commerce platform, but its built-in reporting tools have limitations — especially when it comes to custom analytics, scalability, and real-time visibility. By syncing Shopify to BigQuery, you unlock the ability to query, visualize, and analyze your store’s data using tools like SQL, Looker, or custom dashboards — without waiting on exports or stale reports.

Key Benefits of a Shopify–BigQuery Integration:

• Real-Time Order Tracking - Stay on top of incoming orders as they happen. With real-time syncing, operations teams can react instantly to spikes in demand, fulfillment bottlenecks, or inventory shortages.
• Unified Customer & Sales Analytics - Combine Shopify data with other sources, like ad platforms, CRMs, or email tools, inside BigQuery to build a complete customer journey. Understand which marketing channels drive the most valuable customers.
• Inventory & Fulfillment Intelligence - Track stock levels, shipping times, and fulfillment rates in near real-time to optimize operations. Build predictive dashboards to forecast low-stock items before they affect revenue.
• Custom, Scalable Reporting - Unlike Shopify’s out-of-the-box reports, BigQuery gives you full control over your data model. Run custom cohort analyses, segment customers by behavior, or generate daily KPI rollups — all in SQL.
• Lower Latency vs. Batch ETL - Traditional ETL tools update data in hourly or daily batches. With Estuary Flow, you get low-latency pipelines that reflect changes in Shopify with minimal delay — ideal for fast decision-making.

In short, syncing Shopify data to BigQuery is essential for any team that wants to move beyond static reports and build a flexible, real-time data stack.

Overview of the Estuary Flow Architecture

Estuary Flow makes it easy to move data from Shopify to BigQuery through a modular, real-time pipeline made up of three core components:

1. Capture

The Shopify connector uses Shopify’s GraphQL Admin API to pull data via bulk queries. It continuously streams resources like products and orders into Flow in near real time.

2. Collection

Captured data lands in a Flow collection, which acts as a streaming data lake with enforced schemas and built-in change tracking.

3. Materialization

The BigQuery connector writes data from collections to your BigQuery tables. It uses Google Cloud Storage as a staging layer, ensuring reliable and efficient delivery.

Key Advantages of This Architecture

• Streaming-First: Unlike batch ETL, Flow processes and materializes data in real-time.
• Schema-Enforced: Built-in validation ensures data consistency from source to destination.
• Scalable & Modular: Add more sources or destinations without rewriting pipelines.
• Low Maintenance: Once configured, pipelines require little to no operational overhead.

This architecture is especially valuable for high-volume Shopify stores that need timely insights without managing a fleet of scripts, jobs, and scheduled ETL workflows.

How to Set Up a Shopify to BigQuery Integration with Estuary Flow

Whether you're a data engineer or analyst, follow these steps to build a real-time, no-code pipeline that streams Shopify data into BigQuery using Estuary Flow.

Step 1: Prepare Your Shopify Store

Before you can capture data with Estuary Flow, you'll need to configure your Shopify store and generate the necessary credentials.

Prerequisites

• A Shopify Admin Account
• An Estuary Flow account
• A Google Cloud project with BigQuery and GCS (we’ll cover this in Step 3)

Create or Access Your Shopify Store

1. Log in to your Shopify Admin account or create a development store via the Shopify Partners dashboard.

2. (Optional) Populate your store with sample data:
   
   
   • Visit Apps > Shopify App Store
   • Install “Simple Sample Data” to add mock products, customers, and orders

(Optional) Generate a Private App and Access Token

You can sign in to Shopify with OAuth to authenticate your connector. However, if you would rather authenticate via an access token, follow these steps to create a new token:

1. In your Shopify Admin dashboard, navigate to:
    Settings → Apps and Sales Channels → Develop apps
2. Click Create an app
3. Under Configuration, grant the required Admin API scopes (see below for a full list)
4. Click Install App
5. Shopify will generate an Admin API Access Token. Save it securely.

Access Token Required Scopes:

plaintext
read_checkouts
read_customers
read_fulfillments
read_inventory
read_locales
read_locations
read_orders
read_products
read_publications

Find Your Store ID

Your Store ID is simply the prefix of your Shopify Admin URL.
For example:

plaintext
https://mystore-dev.myshopify.com/admin → Store ID = mystore-dev

Now that your Shopify store is ready and authenticated, you're all set to capture data in real time using Estuary Flow.

Step 2: Capture Shopify Data in Estuary Flow

With your Shopify store and access token ready, you can now configure Estuary Flow to start capturing real-time data from Shopify's GraphQL Admin API.

Estuary’s native Shopify connector uses Shopify’s bulk query mechanism to continuously ingest data such as products, orders, and customers into Flow collections, which act as schema-enforced, streaming datasets.

Create a Shopify Capture in Estuary

1. Log into Estuary Flow: dashboard.estuary.dev
2. Click “Create Capture”: Choose Shopify (GraphQL) as your source connector.

3. Fill in the required fields:
   
   
   • Store: Your Shopify Store ID (e.g. mystore-dev)
   • Authentication: Authenticate with OAuth or paste in your Admin API access token
     

3. Under Bindings, select the Shopify resources you want to sync.
    See the full list of supported data streams.
   
4. Click Next → Save and Publish

 What Happens Next?

• Estuary Flow will begin capturing Shopify data in real time.
• A Flow collection will be created for each resource you selected (e.g. mystore/products).
• You can inspect the collection’s schema, documents, and change history via the web UI or CLI.

Note: Shopify allows only one active bulk query operation at a time. If another app is running a bulk export, Flow’s capture may be delayed.

Your Shopify data is now continuously streaming into Estuary. Up next: we’ll connect this collection to BigQuery for analytics and dashboarding.

Step 3: Materialize Shopify Data to BigQuery

Now that your Shopify data is streaming into a Flow collection, the next step is to push that data into Google BigQuery, where it becomes queryable and ready for analytics.

Estuary’s BigQuery connector handles this by using a Google Cloud Storage (GCS) bucket as a staging layer, ensuring efficient and reliable delivery of data with exactly-once guarantees.

Prerequisites

Before setting up the materialization, ensure you have:

• A Google Cloud Project
• A BigQuery dataset in your desired region
• A GCS bucket in the same region as the dataset
• A Service account with the following IAM roles:
  • roles/bigquery.dataEditor
  • roles/bigquery.jobUser
  • roles/bigquery.readSessionUser
  • roles/storage.objectAdmin
• A generated JSON key file for that service account

Configure BigQuery as a Destination

1. In Estuary, go to the Collections tab and locate your Shopify collections (e.g. mystore/products).
   
2. Click Materialize and search for the BigQuery connector.

3. Fill out the configuration fields:
   
   
   • Project ID: Your GCP project ID
   • Dataset: The BigQuery dataset to write to (e.g. shopify_data)
   • Region: Region matching your GCS bucket
   • Bucket: Name of the GCS bucket
   • Bucket Path: (Optional) Folder prefix inside the bucket (e.g. shopify/)
   • Service Account Credentials: Paste the full contents of your .json key or upload the secret from a file
     

3. Under Bindings, link your Shopify Flow collections (e.g. mystore/products) to a BigQuery table (e.g. products).
   
4. Optional: Enable delta_updates: true for append-only, event-style tables.
   
5. Click Next → Save and Publish

Data Flow in Action

Once published:

• Flow will write new documents from your collection to BigQuery automatically.
• You’ll see the data appear in your specified tables, typically within seconds of capture.
• No Airflow DAGs or cron jobs required — it’s fully managed and real-time.

Start Syncing Shopify Data to BigQuery Now → Try Estuary Flow Free

Step 4: Query Shopify Data in BigQuery

With your Shopify data now streaming into BigQuery, you're ready to start analyzing it. BigQuery's powerful SQL engine lets you build dashboards, run cohort analyses, and combine your Shopify data with other sources like marketing spend or customer support metrics.

Example Query: Product Inventory Summary

Here’s a basic SQL query to analyze the current product inventory:

plaintext
SELECT
  p.vendor AS vendor,
  COUNT(p.id) AS total_products,
  MAX(p.updated_at) AS last_updated
FROM
  `shopify_data.products` p
GROUP BY
  vendor
ORDER BY
  total_products DESC;

This query helps you understand:

• Which vendors offer the most products
• When the catalog was last updated

Use Cases You Can Build On

• Real-time inventory dashboards
  Track product availability, stock levels, and supplier contributions.
  
• Customer segmentation
  Combine Shopify order and customer data to build behavior-based cohorts.
  
• Revenue tracking
  Join orders with products to calculate revenue per product or category.
  
• Marketing attribution
  Sync Shopify with other tools in Estuary (e.g., Meta Ads, Klaviyo) for full-funnel analytics in BigQuery.

Pro Tip: Use BI Tools

Connect BigQuery to tools like:

• Looker
• Tableau
• Google Data Studio
• Metabase

This enables real-time dashboards that update as new orders or product changes stream into your database via Estuary.

Looking for a similar pipeline but using Snowflake instead of BigQuery? Explore our Shopify to Snowflake integration guide for another real-time workflow.

Conclusion

Syncing Shopify to BigQuery is one of the most effective ways to unlock real-time, scalable e-commerce analytics — but doing it manually or with traditional batch ETL tools can be slow, brittle, and operationally expensive.

Estuary Flow provides a powerful alternative: a streaming-first data integration platform that connects Shopify’s GraphQL API directly to BigQuery using low-latency pipelines and schema-enforced collections.

With just a few steps, you can:

• Capture Shopify data in real time
• Materialize it into fully queryable BigQuery tables
• Build dashboards and insights that reflect your latest business activity
• Skip complex infrastructure and long development cycles

Whether you're an engineer automating pipelines or an analyst needing fresh insights, Estuary helps you go from raw data to actionable reporting in minutes, not days.

Ready to get started?

• 👉 Sign up for Estuary Flow — it’s free to start
• 📚 Explore our Docs
• 🧑‍💻 Need help? Contact our team or request a demo