Distributed Memoization: Avoid Repeating Expensive Work
Some operations produce the same output when given the same input.
Without caching, every request still reaches the service, accesses its dependencies, and performs the same work again. This happens even when the result was calculated moments ago.
I encountered this in a reporting service. The main database query was already accelerated by a materialized view, but every request still had to retrieve, validate, process, and enrich the data.
Caching the final result was faster and cheaper. Instead of optimizing one dependency, we could skip the entire operation.
That experience changed how I think about caching: sometimes the most useful thing to cache is not the data, but the completed work.
This is a form of distributed memoization.
Caching the operation
Traditional caching is often described in terms of database records or query responses.
That helps with data access, but the application may still need to perform the same processing afterward. Distributed memoization looks at the problem differently. It stores the output of an operation.
Consider an operation that generates a sales report:
generateSalesReport({
companyId: "company-123",
period: "2026-07",
currency: "USD"
});
Generating the report may still require retrieving the precomputed data, validating it, applying business rules, enriching it with data from other sources, and formatting the response.
The database query was already fast, but the complete operation still had a cost. If multiple users requested the same report, storing the final result avoided repeating the entire processing pipeline.
The consumer checks a result store first. On a hit, it returns the completed report. On a miss, it calls the original service, which generates and stores the result.
Identifying a result
The result can be stored using a key derived from the operation and its input:
result:sales-report:v3:7f83b165...
A practical key contains:
{operation}:{function-version}:{input-hash}
The function version prevents a new implementation from returning results produced by older code.
The input should be normalized before hashing. Normalization means converting equivalent inputs into a single, consistent representation.
For example, these objects contain the same data:
{"companyId":"company-123","period":"2026-07","currency":"USD"}
{"currency":"USD","period":"2026-07","companyId":"company-123"}
A direct hash of their serialized forms could produce different keys because the fields appear in a different order. Normalization can sort object fields and standardize dates and casing.
After normalization, equivalent inputs produce the same hash and reuse the same stored result.
One possible implementation
The result store could be Valkey, Redis, or another key-value store with suitable latency and expiration support.
In a serverless architecture, the flow might look like this:
The consumer checks Valkey.
On a miss, it calls API Gateway.
API Gateway invokes Lambda.
Lambda reads any required data and calculates the result.
Lambda stores the result in Valkey.
Future requests read it directly.
The same pattern works without serverless components. Lambda could be a service running in a container, a background worker, or an application process. Similarly, the data source could be a database, an external API, a file, an AI model, or another dependency used by the operation.
The important distinction is between the fast path, which returns completed work, and the computation path, which produces results that do not exist yet.
Keeping writes centralized
Consumers only need read access to the result store.
The service responsible for the operation remains the only writer. This prevents consumers from publishing arbitrary results and keeps the calculation logic in one place.
A shared client library can hide the lookup and fallback behavior:
const result = await generateSalesReport(input);
Internally, the library:
Builds the key.
Checks the result store.
Returns the stored value on a hit.
Calls the service on a miss.
It can also handle serialization, timeouts, and fallback consistently across consumers.
Invalidating results
An operation may depend on data that changes over time.
The sales report, for example, may use conversion rates to present transactions in a selected currency. When a rate changes, the application should explicitly invalidate reports calculated with that currency.
async function updateConversionRate(input) {
const conversionRate =
await conversionRateRepository.update(input);
await reportCache.invalidateByCurrency(
conversionRate.targetCurrency
);
return conversionRate;
}
The invalidation can happen directly in the write flow or through an event handler triggered after the data changes.
invalidateByCurrency can use an index that associates each currency with the cached reports that depend on it.
The next request will miss the cache, generate the report again, and store the updated result. A TTL can still be used to remove results that are no longer requested, but it is a secondary mechanism rather than the main invalidation strategy.
When it fits
Distributed memoization is a good fit when:
The operation is deterministic.
The same inputs appear frequently.
Results are relatively small.
The operation or its dependencies have a meaningful cost.
Possible use cases include pricing calculations, reports, data transformations, external API calls, machine-learning inference, and business rules engine evaluations.
I would not add this to a cheap operation or one where inputs rarely repeat. In those cases, the result store becomes another dependency without avoiding enough work to justify it. The pattern becomes useful when the hit rate is high enough for a lookup to regularly replace an entire chain of computation and network calls.