Caching Methods
Choosing between exact, normalized, tiered, and semantic caching
Caching covers the mechanics of cachedCall: cache lookup,
concurrent-miss coalescing, and the CacheAdapter interface itself. These guides are about a
narrower question that page only touches on: once you have caching wired up, which kind of
matching should the cache actually do?
By default, cacheKey is matched exactly, two calls only share a cache entry if their keys are
identical strings. That's the right choice for most cases (it's predictable and needs no extra
dependencies), but it means near-duplicate inputs miss the cache and trigger another LLM call. The
guides in this section cover the built-in and DIY options for closing that gap, in roughly
increasing order of what they can catch and what they cost:
Normalized Caching
Match keys that differ only in case, whitespace, or punctuation, using the built-in
NormalizedCacheAdapter, no external dependencies or network calls.
Tiered Caching
Combine a fast local cache with a shared backing store using TieredCacheAdapter, this one's
about where entries live, not how keys match, and composes with the other two.
Semantic Caching
Match prompts by meaning instead of exact text, using a custom adapter built on resolveKey
plus an embedding function you provide. Not a package export, this is a worked example you
adapt, since it needs an injected embedder and a vector store that fits your scale.
Picking one
- Only formatting varies (case, punctuation, whitespace) between otherwise-identical prompts →
NormalizedCacheAdapter. Free, no dependencies, start here. - Wording or phrasing varies, not just formatting ("capital of France" vs "what's France's capital city") → semantic caching. Costs an embedding call per lookup; see the tradeoffs in that guide before reaching for it.
- Lookups are too slow or too expensive against a single store (e.g. every lookup is a network
round trip to Redis) →
TieredCacheAdapter, layered in front of whatever matching adapter you're already using.
These aren't mutually exclusive. TieredCacheAdapter in particular is meant to be combined with
one of the matching adapters, see Tiered Caching
for how the layering works.
All of this builds on one mechanism: the optional resolveKey hook on CacheAdapter. See
Non-exact matching with resolveKey in
the core Caching page for how it fits into cachedCall itself.