Performance
Caching
Caching Strategies
That Actually Work
Cache invalidation, TTL strategies, stale-while-revalidate, and multi-layer caching that reduced our latency by 80%.
"There are only two hard things in Computer Science: cache invalidation and naming things." The quote is a joke, but cache invalidation genuinely ruins systems when done wrong.
Here's how we cache 2M+ requests monthly with consistent freshness and zero stale data incidents.
The Caching Hierarchy
Multi-Layer Cache Architecture
🚀 Edge Cache (CF)
<1ms • Automatic
↓
⚡ Workers KV
~10ms • Global
↓
📦 D1 Database
~20ms • Regional
↓
🌐 External API
100-500ms • Origin
Pattern 1: Stale-While-Revalidate
Serve cached data instantly, refresh in the background:
stale-while-revalidate.ts
interface CachedData<T> {
data: T;
cachedAt: number;
staleAt: number; // When to start background refresh
expiresAt: number; // When cache is truly invalid
}
async function getWithSWR<T>(
key: string,
fetcher: () => Promise<T>,
options: { staleMs: number; maxMs: number },
env: Env,
ctx: ExecutionContext
): Promise<T> {
const cached = await env.KV.get(key, 'json') as CachedData<T> | null;
const now = Date.now();
// Case 1: Fresh cache - return immediately
if (cached && now < cached.staleAt) {
return cached.data;
}
// Case 2: Stale but valid - return stale, refresh in background
if (cached && now < cached.expiresAt) {
ctx.waitUntil(refreshCache(key, fetcher, options, env));
return cached.data;
}
// Case 3: Expired or missing - must fetch
return await refreshCache(key, fetcher, options, env);
}
async function refreshCache<T>(
key: string,
fetcher: () => Promise<T>,
options: { staleMs: number; maxMs: number },
env: Env
): Promise<T> {
const data = await fetcher();
const now = Date.now();
await env.KV.put(key, JSON.stringify({
data,
cachedAt: now,
staleAt: now + options.staleMs,
expiresAt: now + options.maxMs
}), {
expirationTtl: Math.ceil(options.maxMs / 1000)
});
return data;
}
Pattern 2: Cache Keys That Scale
cache-keys.ts
// BAD: Collision-prone, hard to invalidate
const badKey = `property-${id}`;
// GOOD: Namespaced, versioned, tenant-isolated
function buildCacheKey(parts: {
namespace: string; // e.g., 'property', 'user', 'valuation'
version: string; // e.g., 'v2' (for schema changes)
tenant?: string; // For multi-tenant isolation
id: string;
variant?: string; // e.g., 'full', 'summary', 'minimal'
}): string {
const segments = [
parts.namespace,
parts.version,
parts.tenant,
parts.id,
parts.variant
].filter(Boolean);
return segments.join(':');
}
// Examples:
// property:v2:tenant123:prop456:full
// user:v1:user789:summary
// valuation:v3:tenant123:prop456
Pattern 3: Cache Invalidation
cache-invalidation.ts
// Strategy 1: Direct key deletion
async function invalidateProperty(propertyId: string, env: Env) {
await Promise.all([
env.KV.delete(`property:v2:${propertyId}:full`),
env.KV.delete(`property:v2:${propertyId}:summary`),
env.KV.delete(`valuation:v3:${propertyId}`)
]);
}
// Strategy 2: Tag-based invalidation (via index)
async function invalidateByTag(tag: string, env: Env) {
// Fetch keys with this tag from index
const keysJson = await env.KV.get(`tag:${tag}`);
if (!keysJson) return;
const keys: string[] = JSON.parse(keysJson);
await Promise.all(keys.map(k => env.KV.delete(k)));
await env.KV.delete(`tag:${tag}`);
}
// Strategy 3: Version bump (instant global invalidation)
async function bumpCacheVersion(namespace: string, env: Env) {
const newVersion = `v${Date.now()}`;
await env.KV.put(`version:${namespace}`, newVersion);
// All old keys become orphaned, auto-expire via TTL
}
TTL Guidelines
| Data Type | Stale After | Expire After | Strategy |
|---|---|---|---|
| Static (docs, images) | 1 hour | 24 hours | Long TTL + manual purge |
| Config / Settings | 5 min | 1 hour | SWR + event invalidation |
| Product info | 5 min | 30 min | SWR + webhook invalidation |
| Prices / Availability | 30 sec | 2 min | Short TTL, no SWR |
| User sessions | N/A | 15 min | Sliding expiration |
| AI responses | 1 hour | 24 hours | Hash-based keys |
The Cache Stampede Problem
When cache expires and 100 requests hit simultaneously, all 100 hit the origin. Solution: Use a lock (Durable Object) so only one request fetches, others wait for cache. Or use probabilistic early revalidation—refresh slightly before expiry.
Pattern 4: Stampede Prevention
stampede-prevention.ts
async function getWithLock<T>(
key: string,
fetcher: () => Promise<T>,
env: Env
): Promise<T> {
// Try cache first
const cached = await env.KV.get(key, 'json');
if (cached) return cached as T;
// Try to acquire lock
const lockKey = `lock:${key}`;
const lockId = crypto.randomUUID();
const acquired = await env.KV.put(lockKey, lockId, {
expirationTtl: 10, // Lock expires in 10s
metadata: { ifNoneMatch: '*' } // Only if key doesn't exist
}).then(() => true).catch(() => false);
if (acquired) {
// We have the lock - fetch and cache
try {
const data = await fetcher();
await env.KV.put(key, JSON.stringify(data), { expirationTtl: 300 });
return data;
} finally {
await env.KV.delete(lockKey);
}
} else {
// Someone else is fetching - wait and retry
await sleep(100);
return getWithLock(key, fetcher, env);
}
}
Caching Checklist
- Cache keys are namespaced and versioned
- TTLs match data freshness requirements
- Stale-while-revalidate for latency-critical paths
- Invalidation strategy for each data type
- Stampede prevention for high-traffic keys
- Monitoring for cache hit rates and miss latency
- Tenant isolation for multi-tenant data
- Fallback behavior when cache is unavailable
Good caching isn't about speed—it's about consistency. The fastest cache is worthless if users see stale data. Design for correctness first, then optimize for performance.