Skip to content

Caching & staleness

Derivations are cached and recomputed incrementally, and served with stale-while-revalidate semantics (RFC 5861). This is fully local and open source, with no managed backend required.

How staleness is detected

Each derivation has a data version:

data_version = hash(code, parameters, input versions)
  • File inputs are fingerprinted by content hash (cheap; no parse).
  • SQL inputs are fingerprinted by a freshness probe (e.g. max(updated_at)) when declared, else by hashing the result set.
  • Derivation inputs contribute their output version, composed Merkle-style.

If a cached result exists for the current data version (and is within any TTL), it is reused: no recompute. Change an input, and the data version changes, so the derivation recomputes exactly when it needs to.

Early cutoff

A derivation exposes an output version (a hash of its output) to its consumers. So if a parent recomputes but produces the same output, its children are not invalidated; a step that filters out an unchanged column won't retrain a model downstream of it.

Per-derivation cache policy

from elbi_core import cache, derivation, serve


@derivation(serve=serve.json(), cache=cache.auto(ttl=3600))
def daily_rollup(ctx): ...


@derivation(serve=serve.text(), cache=cache.never())  # reads wall-clock / an API
def live_status(ctx): ...
Builder Behavior
cache.auto() (default) Content-addressed caching. Reuse while code, params, and inputs are unchanged.
cache.auto(ttl=N) …plus serve stale + refresh in the background after N seconds (bounded staleness for inputs you can't cheaply fingerprint).
cache.auto(expire=M) A hard ceiling: never serve a value older than M seconds; the next read blocks for a fresh recompute. Bounds stale-if-error. Must be >= ttl.
cache.auto(deterministic=False) Still cached and reused, but never drives early cutoff (for outputs whose bytes vary across runs, e.g. a trained model).
cache.auto(code_version="v2") Pin the logic version explicitly (busts the cache on bump). The escape hatch for changes auto-detection can't see (see below).
cache.never() Always recompute. For derivations that read untracked state.

How code changes are detected

code_version hashes the derivation's own logic (normalized AST, insensitive to comments and reformatting) and recurses into the user-defined helper functions it calls (and closures/nested functions), so editing a helper busts the cache. Third-party and stdlib callables are treated as opaque (identified by name).

It cannot see everything, so document the blind spots and use code_version="…" to defeat them:

  • calls resolved by attribute (obj.method()) or passed as values (callbacks, registries);
  • runtime-mutable module globals read inside the function;
  • a library upgrade changing a dependency's behavior (an environment change, out of scope);
  • comparisons are valid only within one Python minor version.

Safety: signed cache blobs

Cached values are pickled. The store HMAC-signs every blob and verifies the signature before unpickling (constant-time compare), so a tampered or foreign blob is rejected, never executed. A local cache auto-generates a private key (0600) under the cache dir; a shared or remote store must be given an injected secret so all readers and writers agree.

Caching is sound for any derivation that is a pure function of its declared inputs and parameters. A derivation that reads untracked state (wall-clock, a random seed, an external API) must declare cache.never().

Internal (non-served) derivations

A derivation with no serve contract is internal: it is a node in the graph that other derivations consume, but it is not exposed to agents over MCP. Use it for intermediate steps (a cleaned table, a fitted model, an embedding index) without exposing them to agents:

@derivation(inputs={"sales": Dataset("sales")})  # no serve → internal
def cleaned_sales(ctx): ...


@derivation(inputs={"clean": cleaned_sales}, serve=serve.table())
def report(ctx): ...  # served; consumes the internal node

Serving: stale-while-revalidate

elbi mcp (and serve) serve derivations with the ISR contract: an agent gets the last-good answer immediately, and an out-of-date derivation refreshes in the background. Concurrent refreshes are de-duplicated, and a failed refresh keeps serving the last-good value (stale-if-error). So an expensive derivation never makes an agent wait once it has been computed at least once.

Three lifetime windows: within ttl the cached value is served as-is; past ttl it is served stale while refreshing in the background; past expire it is a hard miss: the next read blocks for a fresh recompute and never serves something older. expire is what stops a perpetually-failing refresh from serving stale forever.

The cache CLI

elbi cache status     # where the cache lives + how many entries
elbi cache clear      # clear everything
elbi cache clear --tag <tag>   # invalidate only tagged entries
elbi cache clear --derivation <name>   # invalidate one derivation's entries
elbi cache gc         # reclaim orphaned blobs (mark-and-sweep)
elbi cache gc --max-age-days 30   # also evict entries older than 30 days

The cache lives on disk under .elbi/cache (an action cache plus a content-addressable blob store) and is gitignored.

Invalidating a running app's cache

A served app invalidates its own cache over HTTP, so a deployment's cache can be busted without a shell on its host:

curl -X POST https://elbi.example.com/api/cache/invalidate \
  -H 'Content-Type: application/json' \
  -d '{"tag": "finance"}'          # or {"derivation": "revenue"}, or both
# {"removed": 2}

With both fields, entries carrying either are removed. A derivation the caller cannot reach answers 404, whichever field named it, the same answer its own page gives. Each tag invalidated is written to the audit log as cache.invalidate. Clearing a deployment's whole cache is not offered over HTTP; that stays a decision made on the host.

The cache is local to each process. On a deployment running several replicas the request clears the replica that answered it, and rolling the deployment is the reliable way to clear them all.

elbi cache clear does the same when pointed at the app with --url or -t <target>, sending --token or ELBI_API_KEY as its credential:

elbi cache clear --tag finance --url https://elbi.example.com

The route is no more protected than the rest of the API: the app ships unauthenticated, so a deployment reachable by anyone but you needs something that authenticates in front of it.

Garbage collection

Each cached result is tagged with its derivation's name, so removing a derivation (elbi cache clear --tag <name>, or deleting an agent-authored one) purges its results. cache gc is a mark-and-sweep over the two layers: it keeps every blob an action-cache record references and reclaims the rest, so orphans left by deletes, evictions, or an interrupted write are collected. A grace window spares just-written blobs, so a concurrent write (blob first, record next) is never reclaimed mid-flight. --max-age-days also evicts entries past an age, a backstop against unbounded growth in a long-running autonomous loop.

Local vs. platform

Everything above runs locally and open-source. The one thing a single local runner can't do is coordinate a cache across multiple instances: that, and push-based (CDC) SQL invalidation, are deliberately out of scope. The cache backend is a pluggable interface (CacheStore), so the platform can supply a shared store without changing any derivation code.