The naive approach
Picture the most direct way to build this: a user asks for AAPL's historical context, and the system computes it right then.
This is the architecture almost everyone reaches for first, because it's how most APIs work — fetch a record, maybe run a light transformation, return it. The problem is what "search it against a database of historical price paths" actually requires once the search isn't limited to AAPL's own history.
What the search actually costs
Konseki's matching isn't single-symbol. A structurally similar setup to AAPL's current price action can show up in a completely different symbol, sector, or decade — the cross-market premise is that all of those are valid evidence, not just AAPL's own past.
That means a single query against one symbol at one lookback period requires scoring that symbol's current price path against every comparable window across the full universe — hundreds of symbols in coverage, each with roughly a decade of daily price history. Each comparison computes seven independent distance components — shape, volatility, trend, range position, volume, risk, and price correlation — before they're combined into a similarity score.
// one symbol, one lookback period
hundreds of symbols
× thousands of historical windows per symbol
× 7 distance components per comparison
× 8 lookback periods served per symbol
// run daily, across the full universe
For one symbol at one lookback period, that's already a non-trivial amount of compute. Konseki's schema serves 8 lookback periods per symbol, and the full universe needs this run daily. Even with multiprocessing across dedicated CPU cores, a meaningful slice of that workload takes hours, not milliseconds — and a live API request needs to return in milliseconds, not hours.
Why caching per-symbol doesn't solve it
A reasonable next idea: compute it once per symbol and cache the result. The problem is that "once" isn't stable. The benchmark condition changes every trading day — yesterday's lookback window is not today's. A cached result from last week describes a setup that no longer exists.
There's nothing stable to cache against. Every trading day produces a new benchmark condition for every symbol, which means every trading day requires a new search — this is a daily cost across the entire universe, not a one-time cost to amortize.
This is the actual reason the engine runs as a batch process after close rather than as a cache warmed once and reused.
The architecture this forces
Once live computation and naive caching are both ruled out, the remaining option is to run the full cross-market search once, for the entire universe, right after each market close — and store the complete output as a permanent artifact rather than something reconstructed on demand.
This is why Konseki's daily snapshots are immutable rather than a live database that gets queried and recomputed. The expensive part — the search itself — happens exactly once per symbol per day, in a batch window after close. What the API actually serves is the output of that batch run, not a live computation triggered by the request.
This also explains a few things about the API surface that might otherwise look like arbitrary design choices:
| What you'd expect from a live API | Why it doesn't apply here |
|---|---|
| A parameter to force live recomputation | There's no live computation to trigger — every response is already a stored result. |
| Historical dates regenerated on request | Historical dates resolve to a stored file, not a recomputed one. |
| A single "as of" timestamp | meta.generated_at and meta.data_through are reported separately, because the output was computed after the fact, not at request time. |
What you get instead
The tradeoff is real: you can't ask for a symbol combination the engine hasn't already computed for that date.
A live-computed system structurally cannot offer that second part, since a live system has no fixed answer to preserve. A query-time system that recomputes on every request can't give you reproducible historical output, because the "current" computation always reflects the current state of the model and universe, not the state on the date you're asking about. A snapshot architecture can — which is also why it supports backtesting without lookahead bias.
When query-time computation would make sense
This isn't a universal argument against live computation — it's specific to this workload. If the search space were small (a handful of symbols, a short history) or the comparison were cheap (a single metric instead of seven weighted distance components across the full universe), live computation would be the simpler and more flexible choice, and pre-computation would be unnecessary overhead.
The reason it doesn't apply here is the combination of breadth and depth: cross-market search means the comparison set is the entire universe, not one symbol's own history, and the seven-component scoring means each comparison isn't a single cheap calculation.
Either constraint alone might be tractable live. Together, across hundreds of symbols updated daily, they aren't — which is the actual reason this is infrastructure rather than a feature flag.