At the start of every month, staff opening the sales aggregation screen start saying, "It froze again." While it displays after waiting a few minutes, during that time, operations slow down even for other staff entering orders.
When you ask for advice on this, the proposal you usually get back is: "Let's build a separate data warehouse for analytics." The direction itself is not wrong. However, decisions that should be made beforehand are often skipped. If you build it without deciding these things, you will end up with higher maintenance costs and operational overhead while the waiting time for your staff remains unchanged.
Why aggregation alone halts operations
Business system databases are optimized for reading and writing single records quickly. Registering a single order, looking up a single customer. This type of processing completes by retrieving only the necessary rows using an index.
Aggregation has the opposite profile. It scans all rows from the past year to calculate totals and counts. When running simultaneously on the same database, the aggregation side monopolizes memory and disk, forcing the normal data entry processing running in the background to wait. This is the source of the symptom where "only the aggregation screen is supposed to be heavy, yet the entire system slows down."
Misunderstanding this, teams sometimes proceed with measures like adding indexes or upgrading server specs. While these can be effective in some cases, the root cause is fundamentally placing them in the same location, so the same symptoms will return as data grows. Incidentally, if a list screen is slow regardless of aggregation, it is often due to a different cause; we have organized how to isolate this in The first place to check when list screens are slow.
There are three choices for placement
The choices are not a binary between "your current DB" and "a cloud DWH." There is a middle ground.
A. Separate within the operational DB. Prepare a single read-only replica and direct aggregations there. The architecture barely changes, and the data entry side is no longer dragged down by aggregations. For many SMB business systems, this is sufficient. Deciding how far you can go with a single general-purpose DB like PostgreSQL is summarized in How far a single DB can take you.
B. Set up a single dedicated analytics DB. Take data extracted daily from the operational DB and store it separately in a format optimized for aggregation. Using a columnar database makes the same aggregation orders of magnitude faster. Some options can even be maintained as files on existing servers without adding extra servers.
C. Subscribe to a cloud data warehouse. Place the data on a platform such as BigQuery. This is effective when you have large data volumes, multiple users exploring freely across departments, or a need to cross-reference external data—when any of these apply. How each department can independently query BigQuery data is covered in Self-service problem-solving with Connected Sheets.
The turning point is not data volume
Trying to decide which to choose based on row count will lead to failure. There are companies that need option C with 1 million rows, and companies where option B is plenty with 100 million rows.
You should look at the following three points:
- Who looks at it? If accounting merely views a fixed report once a month, it is A or B. If sales and manufacturing also want to drill down by changing conditions themselves, you should consider C.
- How fresh does it need to be? If you can make decisions based on "numbers up through yesterday," option B, generated once during the night, is sufficient. If you need the "order status right this second," it is fundamentally a problem to solve on the operational system screens rather than on an analytics platform.
- Who will maintain it? If you choose C, permission management, cost monitoring, and data ingestion maintenance will increase. If a company without dedicated personnel chooses C, six months later they are left with an "infrastructure whose inner workings nobody understands." This is what gets overlooked the most.
Deciding on the third point first reveals that a considerable proportion of companies can fit within the scope of A and B.
Option B is expanding
Among dedicated analytics DBs corresponding to option B, DuckDB is seeing significant movement right now. It is a columnar database specialized for analytical processing that runs directly inside applications or on a single machine without setting up additional servers.
In the preview of the upcoming DuckDB 2.0 released on August 17, 2026, changes that transform its character have been introduced.
- It will be able to run as a server. Previously, it only operated inside a single process, but allowing connections from multiple clients becomes an official feature. Architectures that "place a single small DB for analytics" can now be assembled naturally.
- I/O becomes asynchronous. Concurrency when reading files on cloud storage is increased, which is explained to make queries against remote data up to 20 times faster.
- Types for semi-structured data are introduced. Data in formats like JSON can now be stored efficiently with automatic structural detection.
In published benchmarks, a recursive query traversing a graph with 1 million edges reportedly improved from 4.90 seconds to 0.12 seconds, an approximately 40-fold acceleration.
However, this is a preview. The official release is scheduled for autumn 2026, so it is not something to put into business system production architectures today. What is worth knowing is that "the options between A and C will become much more practical next term." This provides material to re-evaluate whether the currently proposed option C architecture is truly necessary.
Checklist for reviewing proposals
When you receive a proposal to "build a data platform," asking the following three questions reveals its substance:
- If we do not have that platform, what specific problems do we face right now? If nothing comes up beyond "for the future," there is no reason to build it now. If the issue is solely "waiting time at the beginning of the month," there is a high probability it can be resolved with A.
- Who will handle operations, and how much will it cost monthly? Confirm not just the build costs, but also the maintenance costs and who will be responsible. Proposals that leave this blank will stall after being built.
- What happens when we decommission it? Can data be extracted and moved elsewhere? Proposals that have an answer ready for this are usually sound in design as well.
What to do next
First, identify one screen that is reported as slow. Saying "the whole system is heavy" gives you nowhere to start, but narrowing it down to "the monthly sales aggregation screen takes 5 minutes only at the start of the month" significantly narrows your options for solutions.
On top of that, count who views that aggregation and how many times. You will rarely decide to build an entire platform for a report viewed once a month by a single person in accounting. Conversely, if multiple departments view it every day from different angles, the investment in separating its placement will pay off.
At GleamHub, we assist with reviewing business system aggregation processes, designing analytics data placement, and providing second opinions on existing proposals through our development, AI, and automation consulting services. Since the optimal architecture varies depending on data volume and the breadth of users, please reach out for an individual consultation via Contact Us.
