You have terabytes of data in your warehouse. Your customers need that data through an API, fresh, fast, at scale. So what do you do?
You build a specialized pipeline.
You set up dbt jobs to materialize views on an hourly schedule. You copy the output to Postgres so you can serve it with lower latency. You wire up monitoring for the sync. You debug it when it breaks at 3 am. You explain to your customers why the data they're seeing is an hour old. You hire someone to maintain all of it.
This is an architecture tax. Not the cost of any single tool, but the compounding cost of moving data between systems that were never designed to work together for the same job. And almost every team building data-powered products is paying it right now.
We paid it too
At our previous company, Numia, we process petabytes of blockchain data across dozens of networks. For years, everything landed in BigQuery. About $250K in total cost per year. The analytics use case worked: internal dashboards, research queries, dbt models on hourly schedules. Standard setup.
Then our customers started asking for real-time API endpoints. They needed fresh onchain data powering their own products, not stale hourly snapshots.
So we looked at our options. The dbt jobs updating every hour clearly wouldn't cut it. We tried materializing data into Postgres to get lower query latency, but that just shifted the bottleneck. Postgres could serve fast, but it was still waiting on dbt to refresh the underlying views. The reads were quick. The data was stale. The pipeline between BigQuery and the API became a product in its own right to maintain.
The architecture we had: two systems, multiple pipelines, stale data
This is the pattern the entire industry is stuck in. Fivetran's 2026 Enterprise Data Infrastructure Benchmark found that the average enterprise manages 328 data pipelines, dedicates 53% of engineering time to maintenance, and spends $2.2 million per year just keeping pipelines running. Monte Carlo's research shows organizations averaging 67 data incidents per month. The more systems you connect, the more things break. Every pipeline you build between your warehouse and your serving layer is another thing to monitor, another thing to fix, another thing standing between your team and actual product work.
The second pipeline shouldn't exist
The reason you need a separate pipeline in the first place is that traditional data warehouses can't serve the workload your API requires. Snowflake has a 60 second minimum billing window per warehouse start and a default concurrency limit of 8 queries. BigQuery queries typically take 1-2 seconds, even on small datasets, due to job orchestration overhead. These are fine for an analyst running a report. They are fundamentally inadequate for an API endpoint serving thousands of concurrent users expecting millisecond responses.
So you build a pipeline to copy data somewhere faster. But that pipeline is pure overhead. It exists only because your warehouse can't do both jobs.
A real-time data warehouse can.
What changes when you eliminate the pipeline
When we moved to ClickHouse® and built ObsessionDB on top of it, we stopped copying data between systems. We ingested everything into one database. From that single system, we still ran the same analytics queries and powered the same internal dashboards we had before. But now we also served sub-second API endpoints to customers, querying terabytes of data in real time. Same data, same system, no pipeline in between.
After: one system, no pipelines, fresh data everywhere
The economics shifted in every direction at once. ClickHouse itself is dramatically cheaper than BigQuery for analytical workloads. ClickHouse benchmarks at 10 billion rows show an order-of-magnitude cost-performance advantage over the major cloud warehouses. So we cut the warehouse bill. We eliminated the Postgres serving layer. We removed the pipeline tooling and the engineering time spent maintaining it. We got fresher data, faster queries, and a simpler architecture, all at lower total cost.
This isn't a unique story. The pattern repeats across the industry: teams replace multi-system pipeline architectures with a single real-time analytical database and the savings are massive, not because any one tool was overpriced, but because the pipelines connecting them were the real cost all along.
This is what ObsessionDB is for
We built ObsessionDB because we lived this problem and realized the fix wasn't another tool in the pipeline. It was eliminating the pipeline entirely. And most importantly, doing this at scale and economically. ObsessionDB is a managed, ClickHouse-compatible query infrastructure purpose-built for the workload that sits between your warehouse and your API: latency-sensitive, high-concurrency, analytical queries on fresh data. You ingest once and serve everything.
If your team is running dbt jobs to materialize data into a serving layer, maintaining CDC pipelines from your warehouse to Postgres or Redis or Elasticsearch, or explaining to customers why the data is an hour behind, you're paying the architecture tax. That pipeline between your warehouse and your API is the most expensive infrastructure you own, not because of what it costs to run, but because of what it costs your team to build, maintain, and debug.
It doesn't need to exist.