Cache Stampede Prevention Techniques

It's 3 AM. Your database just crashed. The cause? A popular cache key expired, and 10,000 concurrent requests all tried to regenerate it at once. This is a cache stampede—also called the thundering herd or dog pile effect.

What Is a Cache Stampede?

When a cached value expires under high traffic:

1. Request 1 finds cache empty, starts regenerating
2. Requests 2-10,000 arrive before Request 1 finishes
3. All 10,000 requests query the database simultaneously
4. Database overwhelmed, response times spike, possibly crashes

Technique 1: Cache Locking

Only one request regenerates the cache; others wait or use stale data:

async function getWithLock(key, generateFn, ttl) {
    // Try to get cached value
    let value = await cache.get(key);
    if (value) return value;

    // Try to acquire lock
    const lockKey = `lock:${key}`;
    const acquired = await cache.set(lockKey, '1', { nx: true, ex: 10 });

    if (acquired) {
        try {
            // We have the lock - regenerate
            value = await generateFn();
            await cache.set(key, value, { ex: ttl });
            return value;
        } finally {
            await cache.del(lockKey);
        }
    } else {
        // Another request is regenerating - wait and retry
        await sleep(100);
        return cache.get(key);  // Return whatever is there
    }
}

Technique 2: Request Coalescing

Deduplicate concurrent requests for the same key:

const inflight = new Map();

async function getCoalesced(key, generateFn, ttl) {
    // Check cache first
    const cached = await cache.get(key);
    if (cached) return cached;

    // Check if request already in flight
    if (inflight.has(key)) {
        return inflight.get(key);  // Return existing promise
    }

    // Create promise for this regeneration
    const promise = (async () => {
        try {
            const value = await generateFn();
            await cache.set(key, value, { ex: ttl });
            return value;
        } finally {
            inflight.delete(key);
        }
    })();

    inflight.set(key, promise);
    return promise;
}

Now 10,000 concurrent requests result in 1 database query, not 10,000.

Technique 3: Probabilistic Early Expiration

Randomly refresh cache before it actually expires:

async function getWithEarlyExpiration(key, generateFn, ttl) {
    const data = await cache.get(key);

    if (data) {
        const { value, cachedAt } = JSON.parse(data);
        const age = (Date.now() - cachedAt) / 1000;
        const remaining = ttl - age;

        // Probability of refresh increases as expiration approaches
        // At 80% of TTL, 20% chance to refresh
        // At 95% of TTL, 50% chance to refresh
        const refreshProbability = Math.max(0, 1 - (remaining / (ttl * 0.2)));

        if (Math.random() < refreshProbability) {
            // Refresh in background, return current value
            regenerateInBackground(key, generateFn, ttl);
        }

        return value;
    }

    // Cache miss - regenerate synchronously
    const value = await generateFn();
    await cache.set(key, JSON.stringify({ value, cachedAt: Date.now() }), { ex: ttl });
    return value;
}

Technique 4: Stale-While-Revalidate

Serve stale data while refreshing in background:

async function getWithSWR(key, generateFn, { freshTTL, staleTTL }) {
    const data = await cache.get(key);

    if (data) {
        const { value, cachedAt } = JSON.parse(data);
        const age = (Date.now() - cachedAt) / 1000;

        if (age < freshTTL) {
            return value;  // Fresh - return immediately
        }

        if (age < staleTTL) {
            // Stale but usable - refresh in background
            setImmediate(() => regenerate(key, generateFn, freshTTL, staleTTL));
            return value;  // Return stale value immediately
        }
    }

    // Too stale or missing - must regenerate synchronously
    return regenerate(key, generateFn, freshTTL, staleTTL);
}

Technique 5: Scheduled Background Refresh

For critical data, don't wait for expiration—refresh proactively:

// Refresh popular data every 5 minutes, TTL is 10 minutes
// Data never actually expires in practice

setInterval(async () => {
    const popularKeys = await getPopularKeys();

    for (const key of popularKeys) {
        try {
            const value = await regenerateData(key);
            await cache.set(key, value, { ex: 600 }); // 10 min TTL
        } catch (err) {
            console.error(`Failed to refresh ${key}:`, err);
            // Existing cached value will continue serving
        }
    }
}, 5 * 60 * 1000);  // Every 5 minutes

Choosing the Right Technique

• Locking: Best for expensive regeneration that must complete
• Coalescing: Best for high-concurrency scenarios
• Early expiration: Best for gradual traffic without perfect coordination
• Stale-while-revalidate: Best for user-facing latency-sensitive endpoints
• Background refresh: Best for critical data that must never miss

In practice, combine multiple techniques for defense in depth.