How Forward reduced real-time analytics costs by 50% with Estuary

The Challenge
Forward needed a real-time analytics replacement after Rockset’s deprecation. Traditional ETL tools were too expensive or lacked the required transformation capabilities. Managing many-to-many data routing, real-time transformations, and complex DynamoDB ingestion added significant complexity. PostgreSQL struggled with heavy aggregation queries, impacting performance.
The Solution
Estuary Flow provided a cost-effective, flexible solution with native connectors, real-time transformations, and many-to-many data routing. By decoupling ingestion from the warehouse, Forward seamlessly streamed data from DynamoDB, MySQL, and Snowflake to PostgreSQL, MotherDuck, and BigQuery while optimizing query performance.
The Results
- 50% cost reduction in real-time analytics
- Faster queries by shifting transformations upstream
- Simplified data integration with native connectors
- Future-ready stack enabling seamless workload shifts to new destinations like Iceberg.
About Forward
Location
Austin, TX
Industry
Financial Technology (Fintech)
Goals
Build a cost-effective, high-performance analytics stack capable of handling real-time and transactional data with minimal complexity.
Forward is a leader in embedded payments technology. It provides solutions that enable SaaS companies to monetize payments efficiently. Its platform helps clients double revenue per customer while maintaining complete control and transparency over payment processes.
Overview
Forward, an innovative fintech company specializing in embedded payment solutions, found itself at a pivotal moment when its primary operational analytics provider, Rockset, announced it would no longer be available for public use. In urgent need of a robust replacement, Forward required a solution that could integrate data from multiple sources—such as DynamoDB, MySQL, and Snowflake—while supporting modern analytics tools like MotherDuck, BigQuery for AI, and PostgreSQL (AWS Aurora).
Forward's complex ecosystem needed data movement, real-time transformations, many-to-many data routing, and cost-effective scalability to support real-time dashboards exposed to users.
"Estuary’s collections as a portable semantic layer enables seamless adaptation to the best tools for any workload. It powers real-time pipelines across our operational, analytics, and AI data stores. With Kafka materialization, we unify CDC sources, extending our platform’s flexibility into the broader Kafka ecosystem."
Alex Mays, Principal Engineer, Forward
The Challenge
Replacing Rockset Efficiently
Rockset’s deprecation forced Forward to find an alternative quickly that could match or surpass Rockset’s real-time capabilities without incurring excessive costs. Traditional ETL tools like Fivetran were evaluated but proved too expensive, while Hevo lacked the transformation capabilities Forward needed.
Integration Complexity
Forward's data comes from various sources, including DynamoDB, MySQL, and Snowflake.
Rockset provided the real-time ingestion pipelines as well as the data warehouse. In order to quickly evaluate new warehouse solutions, they first needed to decouple ingestion from the underlying warehouse. Engineering a bespoke ingestion pipeline that could combine disparate datasets and materialize them in multiple target warehouses posed a risk of overrunning the Rockset deadline due to the high degree of complexity and overhead required.
One significant hurdle was the use of a single table design in DynamoDB - where 30 different entities coexist in a single table with distinct column sets. Traditional ingestion methodologies and several vendor solutions struggled to transform these entity records into normalized tables.
Performance Bottlenecks
Running complex analytical queries directly on PostgreSQL (AWS Aurora) introduced delays and resource contention. PostgreSQL struggled with heavy aggregation queries and lacked support for continuous materializations, requiring multiple derivations to optimize performance.
Real-Time Analytics Demand
Forward needed real-time analytics for operational dashboards, partner APIs, and user-facing analytics. Delays or data inconsistencies could disrupt client experiences and business operations.
Why Estuary Flow?
Forward chose Estuary Flow to power its operational analytics for several strategic reasons:
Cost Efficiency
Estuary Flow's lean infrastructure and transparent pricing reduced Forward's real-time analytics costs by over 50% per month compared to Rockset. This cost reduction freed up the budget for other growth initiatives.
Real-Time Transformations
Unlike traditional ETL tools, Estuary Flow enables complex real-time transformations. Forward can move joins and data derivations upstream from PostgreSQL, significantly improving query performance and reducing the load on operational databases.
Many-to-Many Connectivity
Flow’s architecture supports many-to-many data routing, allowing Forward to seamlessly stream data from sources like DynamoDB,MySQL, and Snowflake into multiple destinations, including PostgreSQL, MotherDuck for advanced analytics, and BigQuery for AI workloads.
The Results
- 50% Reduction in Data Movement Costs: Forward cut its data movement costs in half after transitioning from Rockset to Estuary Flow.
- Simplified Data Integration: Native connectors eliminated the need for complex, custom-built pipelines, streamlining data workflows.
- Improved Query Performance: By shifting joins and transformations upstream to Estuary, PostgreSQL was optimized for speed layer workloads.
- Real-Time API Enablement: Real-time data pipelines empowered Forward to expose APIs for users, enhancing their embedded analytics offerings.
- Modular, Future-Ready Data Stack: With Estuary, Forward can seamlessly shift workloads to new destinations like MotherDuck or Iceberg as business needs evolve.
Conclusion
Forward’s adoption of Estuary Flow has transformed its real-time analytics landscape. By replacing Rockset, Forward achieved a 50% reduction in operational analytics costs while significantly simplifying data integration and boosting query performance. With MotherDuck managing heavy aggregation workloads, BigQuery supporting AI initiatives, and PostgreSQL optimized for last-mile operational analytics, Forward is poised for continued growth.
Estuary Flow's flexibility and modular architecture lay the foundation for Forward’s future: a scalable, cost-effective platform that supports the company’s mission to deliver cutting-edge payment solutions with maximum efficiency and minimal complexity.