New Data Planes: Expanded Regional Options
Estuary’s public data planes have expanded to the coasts! You can now manage your data in US East or US West regions in AWS cloud.
Lower Your Latency With a Data Plane Near You
Make Use of Estuary’s Expanded Cloud Options
Estuary has added two new data planes to our public SaaS offering. These data planes use AWS for the cloud provider, in us-east-1 and us-west-2 regions.
Why does the specific cloud provider and region matter?
- Lower Latency: When all of your resources are in the same cloud, data flows between them faster, shaving time off of Estuary’s already-low millisecond latency.
- Control Egress Fees: Cloud providers charge egress fees when moving data out of their services. Sticking with the same cloud for all your data needs can see your egress fees drop to zero.
- Manage Compliance: Stay compliant with regulations that require data to remain within certain regional boundaries.
Choose Your Data Plane
Estuary’s list of public data planes now includes:
- AWS
us-east-1- NEW - GCP
us-central1 - AWS
us-west-2- NEW - AWS
eu-west-1
Newly created tenants and tenants who only use Estuary’s trial bucket for storage will see these new data plane options by default.
Existing tenants on the trial bucket will now use AWS us-east-1 as their default data plane. You can still select other data planes during task creation. If working with the flowctl CLI, you can specify your data plane with the --init-data-plane flag.
Want to make the switch using your own storage bucket?
Drop us a line—we’ll add the new data planes to your existing tenant. And, since we know you’d have chosen those regions originally if they’d been available, we’ll cover your migration costs.
Getting Started
Whether you’re starting fresh or migrating from an existing data plane, you will need to ensure all your resources can talk to each other. First, collect information about your desired data plane:
- On the Admin Settings page, scroll down to the Data Planes table.
- Find the row for your desired data plane(s).
- Note AWS IAM or GCP service account details for storage authorization (see below).
- Note CIDR blocks for IP allowlisting (see below).
Authorize Storage
Estuary stores collection data to your own storage bucket. Any data planes you wish to use will need access to this bucket.
If you use a Google Cloud Storage bucket, add the data plane’s GCP service account email as a principal on your bucket.
If you use an AWS S3 bucket, add the data plane’s AWS IAM User ARN in your bucket policy.
📚 Check out a full guide on setting up your storage mappings.
Still using Estuary’s trial bucket? No need to make any changes—you can automatically use the new data planes with the trial bucket.
Allowlist IPs
Some systems that interact with Estuary (such as MongoDB) require you to allowlist Estuary’s IPs in order to communicate.
Make sure to add the data plane’s IPs to the external service’s allowlist. You can find the relevant IPs from the Estuary dashboard or in our docs.
Choose Task Data Plane
Ready to use your data plane?
When creating a new capture or materialization, make sure to select the right data plane. You can make your choice from the dropdown in the Capture/Materialization Details section.
FAQs
I can’t select the new data planes when creating a new task. How can I use them?
Existing tenants using their own storage buckets don’t automatically have access to the new data planes. We’re happy to add these options to your account—just reach out to us, whether in Slack or email.
Can I use more than one data plane for my pipelines?
Yes! You can choose the data plane that works best for certain data, and easily choose another data plane for a different pipeline. Just make sure any external resources, like storage buckets, authenticate the correct data planes.
I’d like to use a data plane in a region that’s not listed. What are my options?
Estuary chooses some of the most popular regions for our public data planes. If your use case would benefit from a specific cloud or region not on our list, consider requesting a private or BYOC data plane where you have full control over cloud details.
Keep costs and latency low when you use the right data plane for the job.
