Cache Policies, Explained

Caching is the part of a query library where good intentions turn into stale bugs. gpui-query keeps the surface area deliberately small: three CachePolicy variants, each with a clear contract.

pub enum CachePolicy {
    NoCache,
    Ttl { ttl_ms: u64 },
    StaleWhileRevalidate { ttl_ms: u64 },
}

Let's walk through each one and when to reach for it.

NoCache

CachePolicy::NoCache never caches. Every observer mount triggers a fresh fetch. Use it when the data is cheap to recompute and must always be current — live presence, ephemeral session state, or anything where showing even one millisecond of stale data is wrong.

use gpui_query::CachePolicy;

let policy = CachePolicy::NoCache;
assert_eq!(policy.ttl_ms(), None);
assert!(!policy.can_short_circuit());

It is also the safe default when you are unsure: you trade a little performance for never having to reason about staleness.

Ttl

CachePolicy::Ttl { ttl_ms } caches a successful result for ttl_ms milliseconds. While the entry is fresh, new observers short-circuit — they get the cached value immediately without firing a request. can_short_circuit() returns true only for this variant.

let policy = CachePolicy::Ttl { ttl_ms: 60_000 }; // 60s
assert_eq!(policy.ttl_ms(), Some(60_000));
assert!(policy.can_short_circuit());

Ttl is the workhorse for most API data: a list of projects, a user profile, a config document. Pick the TTL from how stale your UI can tolerate the data being. For collaborative data that changes a lot, keep it short; for slow-moving reference data, let it ride for minutes.

Once the TTL elapses the entry is considered stale; the next observer triggers a refetch.

StaleWhileRevalidate

CachePolicy::StaleWhileRevalidate { ttl_ms } is the variant that makes UIs feel instant. It shares the same ttl_ms semantics for freshness, but after the TTL elapses the cache does not just discard the entry: it serves the stale value immediately to the observer and kicks off a background refetch.

let policy = CachePolicy::StaleWhileRevalidate { ttl_ms: 30_000 };
assert_eq!(policy.ttl_ms(), Some(30_000));
assert!(!policy.can_short_circuit());

Note that can_short_circuit() is false here — unlike Ttl, a stale-while-revalidate entry is never silently trusted as fresh. The point is to render optimistically with old data while a new request runs, then swap in the result. This is the right choice for dashboards and feeds where the user would rather see something now and an update a moment later.

How policies interact with the rest of the crate

A cache policy never acts alone. A few interactions worth knowing:

• Retries. A failed fetch follows RetryPolicy (exponential backoff, capped) regardless of cache policy. NoCache + RetryPolicy::no_retries() is the most pessimistic combination; StaleWhileRevalidate + default retries is the most forgiving.
• Request policies. RequestPolicy::LatestWins cancels in-flight requests when a newer one arrives for the same key, so a revalidation triggered by a stale-while-revalidate hit can be cleanly superseded.
• Observers. Multiple observers on the same key share one underlying QueryResource, so a single background revalidation fans its result out to everyone subscribed.
• Persistence. Because CachePolicy is Serialize/Deserialize, cached state round-trips through a QueryPersister implementation with its TTL intact.

Choosing

Start with Ttl for everything that is not obviously transient, switch to StaleWhileRevalidate for lists and feeds you want to feel snappy, and reserve NoCache for data where staleness is a correctness bug. Three variants, clear contracts, and the rest of the crate plays nicely with whichever one you pick.