Caching Strategies

The 30-second version

Here's the running example for the whole topic. An online shop's product page needs 6 database queries: the product, its price, stock, rating, reviews and related items. At about 50 ms each, the page takes 300 ms before it can even start rendering.

You add a cacheCacheA small, fast store of answers you already computed or fetched, so the next request for the same thing skips the slow work. The hard part is knowing when a cached answer is out of date.: after the first visitor, the finished answer is stored in RedisRedisA popular in-memory data server: a fast key–value store and cache that also offers counters, sorted sets, pub/sub channels and streams (an append-only log for messages)., an in-memory data store. Every later visitor gets it back in about 2 ms (counting the trip to Redis over a network; on one machine it's under 0.1 ms, as §03 measures). That's 150× faster, and the database does almost nothing. Then two things happen in production:

• Monday: marketing drops the kettle's price from €49.99 to €39.99. Customers keep seeing €49.99 for ten minutes, and some complain at checkout. The cached copy was staleStale (data)A cached copy that no longer matches the source of truth because the original changed after the copy was made..
• Black Friday, 18:00: the cached homepage expires at peak traffic. Thousands of requests miss at the same moment, all run the same 6 queries, and the database falls over. That's a cache stampedeCache stampedeWhen a popular cached item expires and many requests miss at the same moment, all rush to the database to rebuild it, and the database falls over. Also called thundering herd or dog-piling..

Try it. The cache below starts empty, and "delete the key on write" is off: when the price changes, nobody tells the cache.

You should see the three facts this whole topic builds on:

1. The first load is a miss (300 ms); the next ones are hits (2 ms).
2. After the price change, hits keep serving the old price until the TTL runs out.
3. Ticking "delete the key on write" fixes it. The next load is one slow miss, then hits with the new price again.

What's in this topic

1. Where caches live, from the user's browser to the database's own memory, and how fast each one is (§02).
2. Patterns for reading and writing through a cache: cache-aside, read-through, write-through, write-behind, refresh-ahead. Real Go and JavaScript against Redis and Postgres (§03–04).
3. Keeping it correct: TTLs, deleting keys, versioned keys, and a race that leaves a stale value behind even when you do everything "right" (§05–06).
4. Stampedes and their fixes, measured: 1,000 requests → 1,000 queries, or 1 (§07–08).
5. HTTP caching in depth, the traps that leak data or show users old values, and running caches at scale (§09–11).

Where caches live

When a shopper opens the product page, the request leaves their browser, crosses the internet, enters your data center, reaches one of your app servers, and finally the database. Any stop along the way can keep a copy of the answer. Each one is a cacheCacheA small, fast store of answers you already computed or fetched, so the next request for the same thing skips the slow work. The hard part is knowing when a cached answer is out of date. with different rules.

A few words from that diagram, in plain terms:

• A CDNCDNContent Delivery Network: servers spread around the world that keep copies of your files close to users, so they're served from a nearby city instead of your origin server. is a company running servers in hundreds of cities. It keeps copies of your responses near your users.
• A reverse proxyReverse proxyA proxy that sits in front of your servers (nginx, Envoy, a cloud load balancer): clients connect to it, and it forwards each request to one of your backends. is a server in front of your app servers (nginx, Varnish) that every request passes through.
• The originOrigin (server)Your own server: the one a CDN or proxy forwards requests to when it doesn't have a usable copy. is your own server: where a CDN or proxy goes when it has no usable copy.
• An in-process cacheIn-process cacheA cache that lives inside your program's own memory (a Go map, a JS Map). The fastest kind, but each running copy of the app has its own, so copies can disagree. is just a map inside your running program.
• The database's buffer poolBuffer poolThe database's own cache: recently used pieces (pages) of tables and indexes kept in the database server's RAM, so many queries never touch the disk. Postgres calls it shared_buffers. is the database's own cache in RAM. You get it for free, and it's always correct.

How fast is a hit?

Real numbers help more than adjectives. The purple bars were measured on this machine, with the product-page code from §03. The pink ones are typical figures, so treat them as approximate:

Three things jump out:

1. An in-process map is about 10,000× faster than Redis. Redis is a network hop away; a map is a memory read. That's why hot, tiny, rarely changing data (feature flags, a country list) often lives in-process.
2. On the same machine, Redis barely beats a simple database lookup (42 µs vs 62 µs). A cache in front of a primary-key query buys little. In front of the 30 ms page query it's a 700× win. Cache expensive answers, not cheap ones.
3. The network dominates. Everything inside your data center is under a millisecond or two. Crossing an ocean costs 100+ ms, which is why CDNs exist.

Layers stack

Big sites use several layers at once. A request for the product page might be answered by the browser (no request at all), or else by the CDN, or else by an in-process cache on the app server, or else by Redis, and only then by Postgres. Each layer that answers protects every layer behind it.

The catch is that every layer is another copy that can be stale, with its own TTL and its own way to be cleared. Stale copies can even stack. Say Redis may hold the product for 10 minutes and the CDN keeps whatever the app sent it for 5 more. If nobody deletes the Redis key, the CDN can refill from a 9-minute-old Redis copy and serve it for another 5: up to 15 minutes before everyone sees a new price. (The browser doesn't add its own max-age on top: a CDN sends an Age header saying how old its copy already is, and the browser counts that against its lifetime.) §09 shows how to choose those numbers, and how to purge.

Cache-aside: the default pattern

Cache-asideCache-aside (lazy loading)The most common caching pattern: your code checks the cache first; on a miss it reads the database itself and puts the result in the cache. On a write it updates the database and deletes the cached copy. is called that because the cache sits to the side of your app: your code talks to the cache and to the database separately, and decides when to use each. It's also called lazy loading, since nothing is cached until someone asks for it. It's the pattern you'll see in most codebases.

The read path has three steps:

1. Ask the cache for the keyCache keyThe name a cached value is stored under, like product:42. Two requests share a cached answer only if they produce the same key, so everything the answer depends on must be in it. product:42. A hit means you're done.
2. On a miss, query the database, which is the source of truthSource of truthThe one place whose copy of the data counts as correct, usually the database. Every cache is just a copy of it that may be out of date..
3. Store the answer in the cache with a TTLTTL (time to live)An expiry time attached to a piece of data: “delete this row 7 days after it was written.” Caches and DNS use TTLs so stale entries disappear on their own., so the next reader gets a hit.

Here are the two tables behind our product page. Note the version column: every change to a product adds 1 to it. You won't need it yet, but §05 and §09 build on it. The database bumps it itself, with a trigger: a small function Postgres runs automatically before every UPDATE of the table, so the version moves even when an admin script or a hand-typed SQL query forgets about it.

CREATE TABLE products (
  id          int  PRIMARY KEY,
  name        text NOT NULL,
  price_cents int  NOT NULL,           -- €49.99 is stored as 4999
  version     int  NOT NULL DEFAULT 1  -- +1 on every change
);
CREATE TABLE reviews (id bigserial PRIMARY KEY, product_id int NOT NULL, stars int NOT NULL);

-- Bump version on EVERY update, whoever runs it. App code never touches version.
CREATE FUNCTION bump_version() RETURNS trigger AS $$
BEGIN
  NEW.version := OLD.version + 1;  -- OLD = the row before the UPDATE, NEW = the row being written
  RETURN NEW;                      -- write the (modified) NEW row
END $$ LANGUAGE plpgsql;

CREATE TRIGGER products_bump_version BEFORE UPDATE ON products
  FOR EACH ROW EXECUTE FUNCTION bump_version();

In that SQL, $$ … $$ wraps the function's body, := assigns a value, and BEFORE UPDATE means the function can change the row before it's saved. So the app's UPDATE statements below only set the price; the version takes care of itself.

Here's cache-aside for the product page, against a real Redis 7.0 and Postgres 16. db is a connection poolConnection poolA set of open database (or HTTP) connections that your program keeps and reuses instead of opening a new one per request. Pools usually have a size limit; when every connection is busy, the next caller waits for one to be returned. for Postgres (pgxpool in Go, pg.Pool in Node): a set of open connections that requests borrow and give back. The "slow" query joins one product with its 2 million reviews to compute the average rating:

type Store struct {
    db  *pgxpool.Pool
    rdb *redis.Client
}

func productKey(id int) string { return fmt.Sprintf("product:%d", id) }

// GetProduct is cache-aside: look in Redis, fall back to Postgres, then fill Redis.
func (s *Store) GetProduct(ctx context.Context, id int) (Product, error) {
    key := productKey(id)
    var p Product

    data, err := s.rdb.Get(ctx, key).Bytes() // 1. ask the cache
    if err == nil && json.Unmarshal(data, &p) == nil {
        return p, nil // hit
    }
    if err != nil && !errors.Is(err, redis.Nil) { // redis.Nil = "no such key"; anything else = Redis trouble
        slog.Warn("cache get failed, using the DB", "key", key, "err", err) // don't fail the request
    }

    p, err = s.loadFromDB(ctx, id) // 2. miss: ask the database
    if err != nil {
        return p, err
    }
    data, _ = json.Marshal(p) // can't fail for a plain struct like Product
    s.rdb.Set(ctx, key, data, 10*time.Minute) // 3. fill (best-effort: if it fails, the next read just misses)
    return p, nil
}

func (s *Store) UpdatePrice(ctx context.Context, id, cents int) error {
    _, err := s.db.Exec(ctx, `UPDATE products SET price_cents = $1 WHERE id = $2`, cents, id)
    if err != nil {
        return err
    }
    return s.rdb.Del(ctx, productKey(id)).Err() // ← invalidate AFTER the DB write
}

loadFromDB is the ordinary query you'd write without a cache: one SELECT … LEFT JOIN reviews … GROUP BY. Here's what three calls in a row look like, then a price change:

call 1: db     31.963ms  Blue Kettle €49.99 ★2.83
call 2: cache     106µs  Blue Kettle €49.99 ★2.83
call 3: cache      54µs  Blue Kettle €49.99 ★2.83
after raw UPDATE (no invalidation): cache says €49.99  ← stale, DB says €39.99
after UpdatePrice (DB write + DEL):  db says €39.99

The miss costs the full 32 ms query. Hits cost 50–110 µs, roughly 300–700× faster. (The Node version measured 34 ms for the miss and about 90 µs per hit.)

The fourth line is the important one. Someone changed the price directly in the database, without going through UpdatePrice. The cache had no way to know, so it kept serving €49.99. Cache-aside only stays correct if every write path deletes the key: your API, your admin panel, your nightly import script. §05 covers what to do when you can't guarantee that.

Try the pattern

The widget from §01 again, now with two more buttons. Evict key is the cache throwing out an entry because it ran out of memory. Restart cache empties the whole cache, as a crash or a deploy would:

Why delete, and not update the cache?

On a write, you could also SET the new value into the cache instead of deleting it. Deleting is the usual choice, for three reasons:

1. Two writers can race. Admin A sets €39.99, admin B sets €44.99. The database applies them in one order, but the two SETs can reach Redis in the other order. The cache then holds the loser's price until the TTL. With DEL, both writers just empty the key, and the next reader loads whoever won.
2. The cached value is often not the row. Our cached product includes the average rating, computed from 2 million reviews. To SET it, every price change would have to re-run the expensive query. A DEL costs nothing; the next reader rebuilds it only if someone actually asks.
3. Write-heavy keys waste work. If a value changes 100 times before anyone reads it, updating it 100 times is pure waste.

Deleting isn't perfect either: there's a rarer race between a slow reader and a writer, which §06 steps through. The order matters too: write the database first, then delete. If you delete first, a reader can sneak in between the delete and the database write, reload the old value, and put it right back.

Read-through, write-through, write-behind, refresh-ahead

Try them all with the same script: load, load, change price, load, wait, load. Then press Restart cache right after a price change, especially with write-behind.

Read-through: the cache does the loading

With read-through, callers never touch the database. They call cache.Get(key), and on a miss the cache calls a loader function you supplied once, up front. The logic is the same as cache-aside; what changes is where it lives. It's written once, in one place, instead of in every handler.

// ReadThrough wraps Redis. Callers only ever call Get; the loader is given once.
type ReadThrough[V any] struct {
    rdb  *redis.Client
    ttl  time.Duration
    load func(ctx context.Context, key string) (V, error)
}

func (c *ReadThrough[V]) Get(ctx context.Context, key string) (V, error) {
    var v V
    data, err := c.rdb.Get(ctx, key).Bytes()
    if err == nil {
        err = json.Unmarshal(data, &v)
        return v, err
    }
    if !errors.Is(err, redis.Nil) {
        return v, err
    }
    if v, err = c.load(ctx, key); err != nil { // ← the cache loads it, not the caller
        return v, err
    }
    data, _ = json.Marshal(v)
    c.rdb.Set(ctx, key, data, c.ttl)
    return v, nil
}

// Setup, once:
//   products := &ReadThrough[Product]{rdb: rdb, ttl: 10 * time.Minute, load: loadProduct}
// Every handler:
//   p, err := products.Get(ctx, "product:7")

  loader called for product:7
{7 1999}
{7 1999}

In JavaScript, the same idea is usually a small wrapper such as cached(key, ttlSeconds, () => loadFromDB(id)). Libraries like cache-manager call it wrap. In-process caches in Java (Caffeine) and Go (several libraries) offer read-through as their main API.

Write-through: every write updates both

With write-through, a write updates the database and puts the new value into the cache before it returns. Readers right after a write get a hit with the new value, never a miss.

func (s *Store) UpdatePriceWriteThrough(ctx context.Context, id, cents int) error {
    _, err := s.db.Exec(ctx, `UPDATE products SET price_cents = $1 WHERE id = $2`, cents, id)
    if err != nil {
        return err // DB failed: don't touch the cache
    }
    p, err := s.loadFromDB(ctx, id) // rebuild the full cached value…
    if err != nil {
        return err
    }
    data, _ := json.Marshal(p)
    return s.rdb.Set(ctx, productKey(id), data, 10*time.Minute).Err() // …and store it
}

Look at the cost hiding in the middle: our cached value contains the average rating, so every price change re-runs the 30 ms query. Write-through is a good fit when the cached value is the row you just wrote. It's a poor fit for expensive, derived values.

Two more catches. It still has the two-writer race from §03 (two SETs can arrive in the wrong order), so keep a TTL. And it caches everything written, including things nobody will read, so pair it with a TTL or an LRU limit.

Write-behind: the database catches up later

With write-behind (also called write-back), a write goes to the cache and into a queue, and the request returns immediately. A background worker saves the queue to the database later, often merging many writes into one.

type priceChange struct{ id, cents int }

type WriteBehind struct {
    rdb   *redis.Client
    db    *pgxpool.Pool
    queue chan priceChange // buffered: make(chan priceChange, 1000)
}

// SetPrice answers as soon as the cache has the new value.
func (w *WriteBehind) SetPrice(ctx context.Context, id, cents int) error {
    if err := w.rdb.Set(ctx, "price:"+strconv.Itoa(id), cents, time.Hour).Err(); err != nil {
        return err
    }
    w.queue <- priceChange{id, cents} // ← the DB write happens later. (If the queue is full, this
                                      //   blocks: backpressure. Real code would select on ctx.Done().)
    return nil
}

// flushLoop saves queued changes once per second. Several changes to one product = one DB write.
func (w *WriteBehind) flushLoop(ctx context.Context) {
    ticker := time.NewTicker(time.Second)
    defer ticker.Stop()
    pending := map[int]int{} // product id → newest price
    for {
        select {
        case c := <-w.queue:
            pending[c.id] = c.cents
        case <-ticker.C:
            for id, cents := range pending {
                _, err := w.db.Exec(ctx, `UPDATE products SET price_cents = $1 WHERE id = $2`, cents, id)
                if err != nil {
                    continue // keep it in pending; try again next tick
                }
                delete(pending, id) // (the demo also prints "flushed …" here)
            }
        case <-ctx.Done():
            return // ← anything still in pending is lost
        }
    }
}

5 SetPrice calls took 1.028ms in total
right away: cache=3499 db=4999
flushed product 42 → 3499 (1 DB write)
1.2 s later: cache=3499 db=3499
for comparison, 5 direct DB UPDATEs took 2.338ms

Five writes became one database write, and the database was a second behind the cache. On this machine the database is local, so direct writes are only about 2× slower. Over a network, with replicasReplicaA full copy of a database kept on another machine and updated as the original changes, for reading load and for taking over if the original dies. (copies of the database) confirming every commit, the gap is often 10× or more.

Refresh-ahead: reload before it expires

With refresh-ahead, a read that finds an entry close to expiry still returns it immediately, and also triggers a background reload. Popular keys get refreshed before they ever expire, so readers almost never wait for a miss.

The cost is reloading entries that nobody would have read again, and the background reloads still hit the database. It shines for a small set of hot keys (the homepage, the top 100 products). §08 shows a clever version, probabilistic early refresh, that also prevents stampedes.

Invalidation: TTL, delete, or a new key

There are only two hard things in Computer Science: cache invalidation and naming things.— Phil Karlton (attributed)

Cache invalidationCache invalidationRemoving or updating cached copies when the original data changes, so readers don't keep seeing stale values. Famously one of the hard problems in computing. means making sure readers stop getting an old copy after the original changes. Here are the three tools, using the fridge from §01 once more:

TTL: how stale is OK?

A TTLTTL (time to live)An expiry time attached to a piece of data: “delete this row 7 days after it was written.” Caches and DNS use TTLs so stale entries disappear on their own. is the simplest invalidation there is, and it never forgets. The only question is the number, and the honest answer is "how stale can this be before someone is harmed?" That's a product question, so ask it per piece of data:

• Exchange rates on a travel site: a few minutes is fine.
• Stock count on a product page: 30 s is fine if checkout re-checks the real stock.
• The price charged at checkout: zero. Read it from the database, never from a cache.
• A user's own profile, right after they edit it: zero for them, minutes for everyone else (§10).

A short TTL means fresher data but more misses and more database load. A long TTL means the opposite. A TTL with delete-on-write gets you both: fresh after normal writes, and the TTL caps the damage when a delete is missed.

Delete: the workhorse

You saw it in §03: write the database, then DEL the key. Its weakness isn't the delete itself. It's the word every: every service, admin panel, script and migration that changes the data must also delete the right keys. Forgetting one is the most common cache bug there is. (And there's one subtle race even when nobody forgets: §06.)

New key: versioned keys

A versioned keyVersioned keyA cache key that includes a version number, like product:42:v7. Changing the data bumps the version, so readers simply start using a new key and the old entry is never read again. puts a version number in the key itself. To invalidate, you don't delete anything; you bump the version, and readers start building new keys. Nobody reads the old entries again, and their TTL eventually clears them out.

This is the easy way to invalidate a whole group at once. A category with 40 pages of products, each page cached under its own key, can be retired with a single INCR:

// Every page of one category shares a version number. Bumping it retires them all at once.
func categoryPageKey(ctx context.Context, rdb *redis.Client, cat, page int) (string, error) {
    ver, err := rdb.Get(ctx, fmt.Sprintf("catver:%d", cat)).Int64()
    if errors.Is(err, redis.Nil) {
        ver, err = 0, nil // no version yet: start at 0
    }
    if err != nil {
        return "", err
    }
    return fmt.Sprintf("cat:%d:v%d:page:%d", cat, ver, page), nil
}

func invalidateCategory(ctx context.Context, rdb *redis.Client, cat int) error {
    return rdb.Incr(ctx, fmt.Sprintf("catver:%d", cat)).Err() // old keys are never read again
}

before: cat:7:v0:page:1
after:  cat:7:v1:page:1 → miss: true

The price is one extra Redis lookup per read (to learn the current version). A cheaper variant takes the version from the data itself: our products table has a version column that every update increments (a trigger bumps it on every UPDATE, §03). A primary-key lookup of version costs about 60 µs, so product:42:v7 tells you whether the expensive 30 ms answer you cached is still current. §09 uses that same version number as an HTTP ETag.

Versioned keys also sidestep the §06 race for the old entry: a slow reader can only write to the key it computed, and nobody will ask for that version again.

When writes happen outside your code: invalidate from the database

The weak spot of delete-on-write is "every write path must remember". The robust fix is to stop relying on the app: watch the database's own record of changes, and delete keys from there. Whoever changed the row, through whatever path, the cache hears about it.

The production-grade version is change data capture (CDC)Change data capture (CDC)Reading a database's own change log (in Postgres, the write-ahead log) as a stream of every insert, update and delete, so another system can copy the changes without the app doing anything.: a tool such as DebeziumDebeziumAn open-source change data capture tool: it reads a database's change stream (Postgres logical decoding, MySQL binlog) and publishes each row change as an event, usually to Kafka. reads Postgres's write-ahead logWrite-ahead log (WAL)An append-only file where a database records each change before applying it. After a crash it replays the log to recover; replicas can follow it to copy changes., the database's own journal of every change, and publishes each change to a message brokerMessage brokerA server that accepts messages from producers and hands them to consumers later — RabbitMQ, Kafka, SQS, NATS. It decouples who sends from who handles, and buffers bursts.. An invalidation worker consumes the stream and deletes the matching keys. Replication §04 explains the log being read; Queues covers consuming the stream safely.

Postgres also has a much smaller built-in tool, LISTEN/NOTIFY, that's enough to see the idea. A trigger announces every product change, and a listener deletes the key:

CREATE FUNCTION notify_product_change() RETURNS trigger AS $$
BEGIN
  -- NEW = the row after the change (NULL on DELETE); OLD = the row before it (NULL on INSERT)
  PERFORM pg_notify('product_changed', COALESCE(NEW.id, OLD.id)::text); -- payload: the product id
  RETURN NULL; -- an AFTER trigger's return value is ignored
END $$ LANGUAGE plpgsql;

CREATE TRIGGER product_changed AFTER INSERT OR UPDATE OR DELETE ON products
  FOR EACH ROW EXECUTE FUNCTION notify_product_change();

notified: product 42 changed → deleted 1 key(s)
UPDATE committed by another client
INSERT committed by another client
notified: product 99999 changed → deleted 1 key(s)
DELETE committed by another client
notified: product 99999 changed → deleted 1 key(s)

(The two programs print independently, so a notification can appear just before or after the line about its commit.) The INSERT matters too: if the app had cached "product 99999 doesn't exist" (§10), creating it must clear that.

The stale-set race (and how to close it)

Cache-aside has two kinds of request touching the same key:

• a reader that missed: it reads the database, then writes the cache;
• a writer: it writes the database, then deletes the cache key.

Each is correct on its own. But each takes two steps, and the steps of two requests can interleave. That's a race conditionRace conditionA bug where the result depends on the timing or order in which concurrent tasks happen to run. Run the same code twice, get two different answers.: the result depends on the timing. Here's the bad interleaving. Step through it, then switch to the fixed version:

1// Reader A: GET /products/42
2v, err := cache.Get("p:42") // miss
3if errors.Is(err, redis.Nil) {
4    v = db.LoadPrice(42)  // reads DB
5    cache.Set("p:42", v, 10*time.Minute)
6}
7 
8// Writer B: PUT /products/42
9db.SetPrice(42, 39.99)
10cache.Del("p:42")

Why this happens

The writer's delete is only useful if it comes after the stale value lands in the cache. In the bad order, the delete comes first and finds nothing to delete. The reader then stores the value it read before the write. Nothing is left to clean it up except the TTL.

It needs three things at once: a miss, a write to that same row, and the reader pausing between its database read and its cache write. A pause of a few milliseconds is enough. A garbage-collection pausePause (GC or VM)A moment when a program's runtime freezes all its threads, for example for garbage collection. From outside, a paused process looks exactly like a dead or unreachable one., a slow network packet, or an overloaded server all do it. This is rare per request, but a busy site does millions of requests a day, so it happens.

We reproduced it against a real Redis and Postgres by running the steps in exactly that order:

naive cache-aside:
  start                                        cache=(none) db=4999
  A missed and read 4999 from DB, then paused  cache=(none) db=4999
  B updated DB to 3999 and deleted the key     cache=(none) db=3999
  A resumes and SETs what it read              cache=4999   db=3999
with a lease:
  A took lease (true), read 4999, paused       cache=(none) db=4999
  B updated DB, deleted key + lease            cache=(none) db=3999
  A's fill accepted? 0                         cache=(none) db=3999

Fix 1: a short TTL (always)

The TTL bounds the damage: the stale price lives at most one TTL. For many caches, "rare, and never longer than 60 s" is an acceptable answer, and nothing more is needed. Decide that consciously, though, rather than by accident.

Fix 2: a lease (make the late write conditional)

This is the fix in the stepper. It was popularised by Facebook's memcache paper (2013). On a miss, the reader first takes a leaseLeaseA permission that expires on its own after a set time unless it is renewed: “you are the leader for the next 10 seconds”. If the holder crashes or is cut off, the lease runs out and someone else can take over. Caches use the same idea for “the right to fill this key”.: a marker key holding a random token, written only if no marker exists yet. The writer deletes the value and the lease. The reader's final write goes through a small Lua scriptLua script (Redis)A small program in the Lua language that you send to Redis with EVAL. Redis runs the whole script without running any other command in between, so a multi-step read–decide–write becomes one atomic step. that stores the value only if its lease is still there:

// fillIfLeased: only write the value if our lease is still there.
var fillIfLeased = redis.NewScript(`
if redis.call('GET', KEYS[2]) == ARGV[1] then
  redis.call('SET', KEYS[1], ARGV[2], 'EX', ARGV[3])
  redis.call('DEL', KEYS[2])
  return 1
end
return 0`)

// Reader.
func getPrice(ctx context.Context, id int) (int, error) {
    key, lease := fmt.Sprintf("p:%d", id), fmt.Sprintf("lease:p:%d", id)
    for {
        v, err := rdb.Get(ctx, key).Int()
        if err == nil {
            return v, nil // hit
        }
        if !errors.Is(err, redis.Nil) {
            return 0, err
        }
        token := rand.Text() // crypto/rand, Go 1.24+: a random string
        ok, err := rdb.SetNX(ctx, lease, token, 10*time.Second).Result()
        if err != nil {
            return 0, err
        }
        if !ok { // someone else holds the lease and is loading it: wait, then look again (§07)
            select {
            case <-time.After(50 * time.Millisecond):
                continue
            case <-ctx.Done():
                return 0, ctx.Err()
            }
        }
        price, err := loadPrice(ctx, id)
        if err != nil {
            return 0, err
        }
        // If a writer deleted our lease meanwhile, this stores nothing. We still return
        // the price we read: our read started before the write.
        err = fillIfLeased.Run(ctx, rdb, []string{key, lease}, token, price, 600).Err()
        return price, err
    }
}

// Writer.
func setPrice(ctx context.Context, id, cents int) error {
    _, err := db.Exec(ctx, `UPDATE products SET price_cents = $1 WHERE id = $2`, cents, id)
    if err != nil {
        return err
    }
    return rdb.Del(ctx, fmt.Sprintf("p:%d", id), fmt.Sprintf("lease:p:%d", id)).Err() // ← cancels any reader's fill
}

The lease does double duty: since only one reader at a time can hold it, it also stops many readers from loading the same key at once. That's the stampede problem, coming up in §07.

Fix 3: versions (refuse anything older)

Store the row's version with the cached value, and make the fill a compare-and-set: "store this only if the cache is empty or holds an older version". The writer can then SET the new value with its new version instead of deleting, and a late reader carrying version 6 can't overwrite version 7. It costs a version column and a Lua script, but it also fixes the two-writer race from §03.

Fix 4: delete twice (a common hack)

Some teams delete, then schedule a second delete a moment later (say 500 ms), hoping any slow reader has finished by then. It shrinks the window rather than closing it: a reader paused for longer than the delay still wins. It's cheap and better than nothing, but it's a guess, not a guarantee.

Stampedes: when a hot key expires

Picture the homepage: 200 requests per second, cached for 5 minutes. Rebuilding it takes a 300 ms query. When the key expires, the first request misses and starts the query. During those 300 ms, another 60 requests arrive, find nothing, and start the same query too.

Sixty copies of the same heavy query now run at once, so each one runs slower, so the rebuild takes longer, so even more requests miss. That feedback loop is a cache stampedeCache stampedeWhen a popular cached item expires and many requests miss at the same moment, all rush to the database to rebuild it, and the database falls over. Also called thundering herd or dog-piling. (also called a thundering herdThundering herdMany clients hitting a server at the same moment — e.g. all retrying together right after an outage — which knocks it over again. or dog-pile). It can take a healthy database down in seconds.

Run it. In this simulation the database runs 20 queries at full speed; past that, every query slows down:

With nothing, about 150 queries pile up, and the slowest requests are still waiting a second later. Every strategy in the toggle cuts that to a handful of queries. They differ in who waits and whether anyone sees old data. This section covers the first two; §08 covers the rest.

Fix 1: request coalescing in one process

The simplest fix: inside one app server, if a load for this key is already running, don't start another. Wait for that one and share its result. That's request coalescingRequest coalescingMerging identical requests that are in flight at the same time, so only one does the slow work and the rest share its result. Go's singleflight and a JS map of promises both do this..

In Go, the package for this is singleflightsingleflightA Go package (golang.org/x/sync/singleflight, not the standard library) that merges identical calls made at the same time: only one actually runs, and every caller gets its result. (golang.org/x/sync/singleflight, maintained by the Go team but outside the standard library). In JavaScript, a Map of in-flight promises does the same in a few lines:

import "golang.org/x/sync/singleflight"

var group singleflight.Group

// Cache-aside + singleflight: concurrent misses for the same key share ONE DB call.
func getRating(ctx context.Context, id int) (string, error) {
    key := "rating:" + strconv.Itoa(id)
    v, err := rdb.Get(ctx, key).Result()
    if err == nil {
        return v, nil
    }
    if !errors.Is(err, redis.Nil) {
        return "", err
    }
    res, err, _ := group.Do(key, func() (any, error) { // ← one caller per key runs this
        v, err := ratingFromDB(ctx, id)
        if err == nil {
            rdb.Set(ctx, key, v, time.Minute)
        }
        return v, err
    })
    if err != nil {
        return "", err
    }
    return res.(string), nil
}

Here's a real run: 200 and then 1,000 simultaneous requests for a key that has just expired, against the 2-million-row rating query. Each "query" is a real Postgres query:

plain           200 requests → DB queries:  200, all answered in 3.541s
singleflight    200 requests → DB queries:    1, all answered in 31ms
plain          1000 requests → DB queries: 1000, all answered in 17.543s
singleflight   1000 requests → DB queries:    1, all answered in 34ms

plain         200 requests → DB queries:  200, all answered in 3485 ms
promise map   200 requests → DB queries:    1, all answered in 53 ms

Without coalescing, 1,000 requests ran 1,000 copies of a 30 ms query and the last answer took 17 seconds: the database worked through them about 20 at a time. With coalescing it was one query, and everyone had an answer in 34 ms. Note what "fixed" means here: the wait didn't just shrink for the database, it shrank for every user.

Fix 2: a lock across the whole fleet

Coalescing only works inside one process. With 20 app servers, a stampede still sends 20 queries, one per server. That's usually fine. When even that's too many (a very heavy rebuild, a lot of servers), use a lock in Redis, so only one request in the whole fleet rebuilds:

// releaseLock deletes the lock only if it still holds OUR token.
var releaseLock = redis.NewScript(`
if redis.call('GET', KEYS[1]) == ARGV[1] then return redis.call('DEL', KEYS[1]) end
return 0`)

func getWithLock(ctx context.Context, key string, load func(context.Context) (string, error)) (string, error) {
    for {
        v, err := rdb.Get(ctx, key).Result()
        if err == nil {
            return v, nil // hit
        }
        if !errors.Is(err, redis.Nil) {
            return "", err
        }
        token := rand.Text()
        won, err := rdb.SetNX(ctx, "lock:"+key, token, 5*time.Second).Result() // lock expires on its own
        if err != nil {
            return "", err
        }
        if won { // ← exactly one caller in the fleet gets here
            v, err := load(ctx)
            if err == nil {
                rdb.Set(ctx, key, v, 5*time.Minute) // §08 adds jitter here
            }
            releaseLock.Run(ctx, rdb, []string{"lock:" + key}, token) // Lua: DEL only if still our token
            return v, err
        }
        select { // someone else is rebuilding: wait a bit, then look again
        case <-time.After(50 * time.Millisecond):
        case <-ctx.Done():
            return "", ctx.Err()
        }
    }
}

lock: 500 concurrent misses → 1 load, done in 142ms

The lock expires by itself after 5 s, so a crashed rebuilder can't block the key forever. It's released with the same token-checking Lua trick as the lease in §06: without it, a slow rebuilder whose lock already expired could delete someone else's lock.

The cost is that everyone else polls: they sleep 50 ms and look again, which adds up to 50 ms of waiting and some extra Redis traffic. The lease from §06 is the same mechanism, and solves the stale-set race at the same time.

Jitter, early refresh and serving stale

Jittered TTLs: don't expire everything at once

After a deploy or a cache restart, many keys get filled within the same minute, all with the same TTL. Exactly one TTL later, they all expire within the same minute too, and the database gets the whole cold start again. The fix is jitterJitterA small random amount added to a delay or timer — a retry wait, a reconnect, a connection's maximum age — so that many clients don't all act at the same instant and stampede a server.: give each key a slightly random TTL.

At 0% every key expires in the same 5-second window. Push the slider to 10%: the same 300 misses are spread over a minute. It's one line of code:

// base ± frac, e.g. 5 min ± 10% → somewhere between 4m30s and 5m30s
func jittered(base time.Duration, frac float64) time.Duration {
    return time.Duration(float64(base) * (1 + frac*(2*rand.Float64()-1))) // math/rand/v2
}

rdb.Set(ctx, key, data, jittered(5*time.Minute, 0.1))

Jitter fixes many keys expiring together. It does nothing for one hot key: that key still expires at some moment, with 200 requests a second behind it. For that you need one of the next two.

Probabilistic early refresh (XFetch)

The idea behind XFetchProbabilistic early refresh (XFetch)Each reader, as an entry gets close to expiring, rolls a die; the closer to expiry, the likelier one of them refreshes it early. With many readers, one refresh happens shortly before expiry instead of a stampede after it. (Vattani, Chierichetti and Lowenstein, 2015): as a key gets close to expiring, every reader rolls a die to decide whether to refresh it early. Far from expiry, nobody does. Close to expiry, it becomes likely, and with many readers, usually one of them refreshes it just before it expires. With enough traffic, almost nobody meets an expired key. (A key read only now and then can still expire: few readers means few dice rolls.)

Each reader refreshes early when:

now − Δ · β · ln(rand())  ≥  expiry

• Δ (delta) is how long the last rebuild took. A 300 ms rebuild should start refreshing roughly in the last few hundred milliseconds; a 10 s rebuild, earlier.
• ln(rand()) is the natural log of a random number between 0 and 1. That's always negative, usually close to 0, and occasionally large. So −Δ·β·ln(rand()) is "a random head start, usually small".
• β (beta) tunes it: 1 is the default; above 1 refreshes earlier, below 1 later.

So the cache entry must store Δ and its own expiry next to the value. The Redis TTL is set a little longer than that logical expiry, so the value is still there to be read near the end:

const beta = 1.0 // β; rand below is math/rand/v2

type entry struct {
    Value  string  `json:"v"`
    Delta  float64 `json:"d"` // how long the last rebuild took, in seconds
    Expiry float64 `json:"e"` // logical expiry, Unix seconds
}

func getXFetch(ctx context.Context, key string, ttl time.Duration, load func(context.Context) (string, error)) (string, error) {
    raw, err := rdb.Get(ctx, key).Bytes()
    var e entry
    if err == nil && json.Unmarshal(raw, &e) == nil {
        now := float64(time.Now().UnixMicro()) / 1e6
        if now-e.Delta*beta*math.Log(1-rand.Float64()) < e.Expiry { // 1-r: never ln(0)
            return e.Value, nil // not this time: plain hit
        }
        // the die says refresh: fall through (usually just this one caller)
    } else if err != nil && !errors.Is(err, redis.Nil) {
        return "", err
    }
    start := time.Now()
    v, err := load(ctx)
    if err != nil {
        return "", err
    }
    delta := time.Since(start).Seconds()
    e = entry{v, delta, float64(time.Now().Add(ttl).UnixMicro()) / 1e6}
    raw, _ = json.Marshal(e)
    rdb.Set(ctx, key, raw, ttl+10*time.Duration(delta*float64(time.Second))) // Redis keeps it a bit longer
    return v, nil
}

We ran 50 concurrent readers for 6 seconds against a key with a 2-second TTL and a 100 ms load, once with plain cache-aside and once with XFetch (β = 1). In both, the key is filled once at t = 0, before the readers start:

plain   14000 reads in 6 s, TTL 2 s → loads: 100, reads that waited for a load: 100
xfetch  14427 reads in 6 s, TTL 2 s → loads:  5, reads that waited for a load:  5
plain   13900 reads in 6 s, TTL 2 s → loads: 100, reads that waited for a load: 100
xfetch  14262 reads in 6 s, TTL 2 s → loads: 11, reads that waited for a load: 11
plain   13904 reads in 6 s, TTL 2 s → loads: 100, reads that waited for a load: 100
xfetch  14373 reads in 6 s, TTL 2 s → loads:  8, reads that waited for a load:  8

Plain cache-aside: the key expired at 2 s and at 4 s (the run ends at 6 s), and at each expiry all 50 readers missed together, so 100 loads, every time. XFetch: a handful of loads, which varied from 5 to 14 across all our runs, because it's random. Every one of them happened while the old value was still being served to everyone else. Sometimes two readers roll "refresh" within the same 100 ms, so there are a few duplicates; combine it with coalescing (§07) to remove those too.

Stale-while-revalidate: serve the old value, refresh behind

Stale-while-revalidateStale-while-revalidateServing the old cached copy for a short while after it expires, while one background request fetches a fresh copy. Readers never wait; they may see data a little out of date. splits the TTL into two parts:

• Fresh (say 60 s): serve it, nothing else to do.
• Stale (say the next 5 minutes): serve it immediately anyway, and start one background refresh.
• After that, it's gone: the next reader has to wait for a real load.

Nobody waits in the common case. With the refresh coalesced, there's at most one refresh per app server (the Map below lives in one process), until the stale window runs out; after that it's a plain miss again. The price is that, for one refresh time after the fresh period ends, readers get slightly old data. For a homepage or a rating, that's almost always the right trade.

const refreshing = new Map(); // key → promise: coalesces refreshes, like §07

function refresh(key, freshSec, staleSec, load) {
  if (!refreshing.has(key)) {
    const p = (async () => {
      const v = await load();
      const entry = { v, freshUntil: Date.now() + freshSec * 1000 };
      await redis.set(key, JSON.stringify(entry), { EX: freshSec + staleSec }); // Redis keeps it through the stale window
      return v;
    })().finally(() => refreshing.delete(key));
    refreshing.set(key, p);
  }
  return refreshing.get(key);
}

async function getSWR(key, freshSec, staleSec, load) {
  const raw = await redis.get(key);
  if (raw === null) return refresh(key, freshSec, staleSec, load); // nothing at all: wait for it
  const { v, freshUntil } = JSON.parse(raw);
  if (Date.now() >= freshUntil) {
    refresh(key, freshSec, staleSec, load)          // no await: runs in the background
      .catch((err) => console.error("background refresh failed:", err));
  }
  return v; // fresh or a little stale: answer right now
}

t=   0 ms  got v1  in 102.0 ms
t= 500 ms  got v1  in   0.9 ms
t=1200 ms  got v1  in   1.6 ms
t=1250 ms  got v1  in   1.0 ms
t=1400 ms  got v2  in   0.9 ms

Read it line by line. At 0 ms the key is missing, so that one reader waits 102 ms. At 500 ms it's fresh. At 1,200 ms it's past its fresh time, so the reader gets v1 instantly and a refresh starts behind it. At 1,250 ms the refresh is still running (coalesced, so no second load), and the reader still gets v1 instantly. By 1,400 ms, v2 is in.

Now compare all five strategies in the simulator from §07, starting on serve stale and early refresh:

HTTP caching in depth

HTTP §08 introduced the basics: Cache-ControlCache-ControlThe response header that tells browsers, CDNs and proxies whether and how long they may reuse a response: max-age=3600, no-cache, no-store, private…, ETagsETagA response header holding a version label for the content (often a hash), e.g. ETag: "v42". The client sends it back in If-None-Match to ask “has it changed since this version?”. and 304 Not Modified304 Not ModifiedA response with no body meaning “the copy you already have is still current — use it”. Saves sending the same bytes again.. This section goes deeper: how browsers and CDNsCDNContent Delivery Network: servers spread around the world that keep copies of your files close to users, so they're served from a nearby city instead of your origin server. treat the same header differently, and how to pick values for real pages.

The directives, one by one

Build some headers and move the age slider. Watch the browser and the CDN disagree:

The product page preset is the interesting one: public, max-age=60, s-maxage=300, stale-while-revalidate=30. The browser treats it as fresh for 60 s; the CDN for 300 s. The CDN gets the longer time because you can purge a CDN (below), but you can't purge a browser. Short browser lifetimes, longer CDN lifetimes plus purges: that's the usual shape for pages.

ETags from a version number

An ETag lets a cache ask "has it changed since version X?" and get a tiny 304 back instead of the full response. The usual way to make one is to hash the response body. But then you have to build the whole response just to find out it hasn't changed, and that's exactly the slow part.

Our products table has a version column that the database bumps on every update, through a trigger (§03). Make the ETag from that, and a 304 costs one primary-key lookup instead of the full page query:

func product(w http.ResponseWriter, r *http.Request) {
    id, _ := strconv.Atoi(r.PathValue("id"))

    // Cheap check first: the row's version number (a primary-key lookup).
    var version int
    err := db.QueryRow(r.Context(), `SELECT version FROM products WHERE id = $1`, id).Scan(&version)
    if errors.Is(err, pgx.ErrNoRows) {
        http.NotFound(w, r) // 404: it really doesn't exist
        return
    }
    if err != nil {
        slog.Error("load version", "id", id, "err", err)
        w.Header().Set("Cache-Control", "no-store") // never let a cache keep an error
        http.Error(w, "internal error", http.StatusInternalServerError) // 500, not 404
        return
    }
    etag := fmt.Sprintf(`"p%d-v%d"`, id, version)

    w.Header().Set("Cache-Control", "public, max-age=60, s-maxage=300, stale-while-revalidate=30")
    w.Header().Set("ETag", etag)
    if r.Header.Get("If-None-Match") == etag { // simplified: see the note below
        w.WriteHeader(http.StatusNotModified) // 304: skip the expensive part entirely
        return
    }
    p, err := loadProductPage(r.Context(), id) // the expensive part
    if err != nil {
        slog.Error("load page", "id", id, "err", err)
        w.Header().Set("Cache-Control", "no-store")
        http.Error(w, "internal error", http.StatusInternalServerError)
        return
    }
    w.Header().Set("Content-Type", "application/json")
    json.NewEncoder(w).Encode(p)
}

// http.HandleFunc("GET /products/{id}", product)

Here's a real session on a fresh Postgres 16 with the §03 schema. The UPDATE in the middle is deliberately an "out-of-band" one, like an admin script would run: it doesn't mention version at all. The trigger bumps it anyway, so the old ETag stops matching:

$ curl -i localhost:8391/products/42
HTTP/1.1 200 OK
Cache-Control: public, max-age=60, s-maxage=300, stale-while-revalidate=30
Content-Type: application/json
Etag: "p42-v1"
Content-Length: 50

{"id":42,"name":"Blue Kettle","price_cents":4999}

$ curl -i -H 'If-None-Match: "p42-v1"' …
HTTP/1.1 304 Not Modified
Cache-Control: public, max-age=60, s-maxage=300, stale-while-revalidate=30
Etag: "p42-v1"

$ psql -c "UPDATE products SET price_cents = 3499 WHERE id = 42"    # an admin script: no version in sight
$ curl -i -H 'If-None-Match: "p42-v1"' …
HTTP/1.1 200 OK
Cache-Control: public, max-age=60, s-maxage=300, stale-while-revalidate=30
Content-Type: application/json
Etag: "p42-v2"
Content-Length: 50

{"id":42,"name":"Blue Kettle","price_cents":3499}

$ curl -i localhost:8391/products/123456
HTTP/1.1 404 Not Found
Content-Type: text/plain; charset=utf-8
X-Content-Type-Options: nosniff
Content-Length: 19

404 page not found

(Go writes the header name as Etag. Header names are case-insensitive, so caches treat it exactly like ETag.) We also checked the Node handler's error path by closing its database pool: it answered 500 with Cache-Control: no-store and no ETag.

Vary: when the URL isn't the whole key

A shared cache looks up responses by URL. If your response also depends on a request header, the cache must know, or it serves one user's variant to another. The VaryVary headerAn HTTP response header naming request headers that change the response, e.g. Vary: Accept-Encoding. Shared caches then store a separate copy for each value of those headers. header says which request headers matter:

• Vary: Accept-Encoding: gzip and uncompressed copies are stored separately. Almost every server sends this automatically.
• Vary: Accept-Language: a German and a French copy. Fine, but it multiplies the number of copies.
• Vary: Cookie: one copy per cookie value, so effectively one per user. A shared cache gains almost nothing; most CDNs won't cache it at all. If a response depends on who's logged in, use private instead.

Purging a CDN

A CDN can hold a copy for the full s-maxage. When the price changes, you don't want to wait five minutes, so you purge: call the CDN's API and tell it to drop its copies. A purgePurge (CDN)Telling a CDN to drop its stored copies right now, by URL or by tag, through the CDN's API, instead of waiting for them to expire. usually reaches every edge server within seconds.

• By URL: "drop /products/42". Simple, but the product appears on many URLs: its page, its category pages, search results, the homepage.
• By tag: your response carries a header listing tagsSurrogate key (cache tag)A label attached to cached responses (e.g. product-42) so you can purge every response with that label in one call, whatever their URLs., such as Surrogate-Key: product-42 category-7 (Fastly) or Cache-Tag: … (Cloudflare). One purge of product-42 drops every response tagged with it, whatever its URL.

It's the CDN version of delete-on-write: the price-change handler writes the DB, deletes the Redis key and purges the tag. It has the same weakness too: every write path has to remember. So the CDN-driven invalidation is often hooked to the same change stream as §05.

Consistency traps

1. Caching "not found" (and forgetting to un-cache it)

Bots and broken links ask for products that don't exist: /products/999999. Cache-aside only caches found values, so every one of those requests misses and hits the database. A scraper looping through IDs bypasses your cache completely.

The fix is negative cachingNegative cachingCaching the answer “this doesn't exist” (a 404, an empty result), so repeated lookups for missing things don't all hit the database.: cache the answer "doesn't exist" too, with a short TTL:

const notFound = "\x00nf" // a marker no real JSON value can equal

data, err := s.rdb.Get(ctx, key).Bytes()
if err == nil {
    if string(data) == notFound {
        return Product{}, ErrNotFound // cached "no": skip the DB
    }
    // … decode the real value as before
}
// … on a miss:
p, err := s.loadFromDB(ctx, id)
if errors.Is(err, pgx.ErrNoRows) {
    s.rdb.Set(ctx, key, notFound, 30*time.Second) // short: it may be created soon
    return p, ErrNotFound
}

The trap is the other direction: when product 999999 is created, the "doesn't exist" entry must go, or the new product shows as a 404 for 30 s. So the create path must delete the key too, not only update and delete. That's another reason to keep negative TTLs short.

2. One user's data, served to another

A cache keyCache keyThe name a cached value is stored under, like product:42. Two requests share a cached answer only if they produce the same key, so everything the answer depends on must be in it. must include everything the answer depends on. Leave something out, and two different answers share one key. Then whoever fills it first decides what everyone else sees. The classic versions:

• cart:summary instead of cart:summary:user:7: every user sees the first user's cart.
• A product price that depends on the user's country or currency, cached under product:42.
• Cache-Control: public on /me (§09): the CDN serves Ada's account page to Bob.

These aren't just bugs; they're data leaks, and they've caused very public incidents. Two habits help: build keys in one function per kind of data (never by hand in each handler), and never cache personalized responses in shared HTTP caches: mark them private.

3. "I saved it, but it still shows the old value"

A user edits their profile, presses Save, and the page that reloads shows the old name. They try again, now worried. What happened: the write deleted the Redis key, but the page itself came from the browser cache, or a CDN, or an in-process cache on another server, or an async invalidation (§05) that hadn't arrived yet.

The guarantee the user expects is called read-your-writesRead-your-writesA guarantee that after you change something, your own later reads show the change. Other users may still see the old value for a moment.: after I change something, my reads show it. Other users can wait a few seconds; the person who just saved can't. Ways to give it to them:

• Return the new object in the write's response, and have the UI show that instead of reloading.
• Bypass the cache for that user for a few seconds after a write (set a short-lived cookie, and skip caches while it's present).
• For their own data, mark the responses private, no-cache so the browser revalidates every time.

4. A cache in front of read replicas

Many apps send writes to one main database server, the primary, and read from replicasReplicaA full copy of a database kept on another machine and updated as the original changes, for reading load and for taking over if the original dies.: copies of the database that receive every change from the primary a moment later. That moment is replication lagReplication lagHow far a follower is behind the leader: the changes the leader has made that the follower hasn't applied yet. Measured in time (0.2 s) or log bytes. Usually milliseconds; can grow to minutes., usually milliseconds, sometimes seconds. Combine replicas with cache-aside, and a miss can fill the cache from a replica that hasn't caught up:

1. The writer updates the primary and deletes product:42.
2. A reader misses and queries a replica, which still has the old price (it's 200 ms behind).
3. The reader caches the old price, with a full 10-minute TTL.
4. The replica catches up 200 ms later, but the cache keeps serving the old price for 10 minutes.

Without the cache, the user would see old data for 200 ms. With it, for 10 minutes: the cache turned a short lag into a long one. Fixes: for a few seconds after a write, fill that key from the primary (which always has the latest write); or delete the key a second time after the typical lag (§06's "delete twice", which fits better here); or use versions (§06), so an old version can't overwrite a newer one. Replication §06 covers lag itself.

5. Hot keys

Sometimes one key gets a huge share of all traffic: the homepage, a viral product, a celebrity's profile. That's a hot keyHot key / hot spotA single key (or a narrow range) that gets far more traffic than the rest, like a celebrity's profile. Because one key always lives on one shard, that shard overloads while the others idle.. When Redis is split across several servers (Redis Cluster, or sharding done by your client library), each key still lives on one server. And one Redis server handles something like 100,000 simple commands per second (very roughly; it depends on value size and hardware). A big enough hot key maxes out that one node while the others sit idle.

• Put a tiny in-process cache in front (§02's two-level pattern): even a 1-second TTL turns 100,000 Redis reads per second into one per second per app server.
• Keep several copies: store it as home#0 … home#7, and have each reader pick one at random. The load splits eight ways; invalidation has to delete all eight.

6. Big values and the cost of decoding

Cache entries feel free, but every hit sends the value over the network and decodes it. Here's a real measurement of Redis GET plus Go's JSON decoding, for lists of products of growing size:

     1 items,      103 bytes: GET + decode 41µs
   100 items,    10381 bytes: GET + decode 157µs
 10000 items,  1078781 bytes: GET + decode 10.897ms

A 1 MB value costs 11 ms per hit, a third of the 30 ms query it replaced, and over a real network it's worse. Redis also executes commands one at a time, so a big GET keeps other clients waiting while it runs. Cache smaller pieces (one entry per product, not one per 10,000-product category), paginate, or compress. Watch the size of your biggest values (redis-cli --bigkeys lists them).

Staff-level: running caches in production

What's worth caching?

A cache adds work to every miss: one extra round trip to Redis before the real query. It pays off only when, roughly,

hit rate × (cost of the query − cost of a hit)  >  (1 − hit rate) × cost of a hit
                                                   + the cost of being wrong (staleness)

Put our measured numbers in. For the 30 ms page query, a hit costs 0.04 ms: almost any hit rate wins. For the 0.06 ms primary-key lookup, a hit (0.04 ms) saves almost nothing, and every miss gets slower. Caching cheap queries mostly buys you staleness. The best candidates are:

• expensive to compute (aggregations, joins, calls to other services, rendering),
• read much more often than written,
• and allowed to be a bit stale, as decided in §05.

Measure before you cache: find the slowest and most frequent queries (Postgres's pg_stat_statements lists them), and check whether an index or a fix for an N+1 query removes the need entirely. A cache in front of a bad query hides it until the cache is empty.

Measure the hit rate, per kind of key

Redis keeps server-wide counters. Here they are from the Redis used for this topic's demos:

expired_keys:10
evicted_keys:0
keyspace_hits:123592
keyspace_misses:10967

That's a hit rateHit rateThe share of lookups a cache can answer by itself. 90% hit rate = 9 of every 10 requests are found in the cache; the other 1 (a miss) has to go to the slow source. of 123,592 ÷ (123,592 + 10,967) ≈ 91.8%. Useful, but it blends every kind of key together: a 99.9% session cache can hide a 20% product cache. In production, count hits and misses in your app, labelled by key family (product, cart, homepage), as countersCounter (metric)A metric that only ever goes up: total requests, total rejections. Dashboards usually show how fast it grows (per second)., alongside:

• evictions (evicted_keys): rising means the cache is too small for its working setWorking setThe memory a piece of code actively touches during some stretch of time. If it fits in a cache level, the code runs at that level's speed. (the entries in active use);
• memory used vs the limit, and the biggest values (§10);
• latency of cache calls, at the p99p99 (99th percentile)The value that 99% of measurements are below — i.e. how bad the slowest 1% is. Averages hide slow outliers; p99 shows them., not only the average;
• DB queries per second, the number the cache exists to keep down.

How to collect and alert on these is Observability's subject.

Size it, and choose what happens when it's full

Redis's default memory limit is 0, meaning "no limit", and its default policy when a limit is set is noeviction. With noeviction, a full Redis refuses writes with an out-of-memory (OOM)OOM-killed“Out of memory”: the system kills a program that uses more memory than allowed. error. Here's what that looks like with a 2 MB limit and 100 KB values:

OK  (×7)
OOM command not allowed when used memory > 'maxmemory'.  (×33)

For a cache, that's the wrong behaviour: you want old entries thrown out, not new ones refused. Set both:

maxmemory 4gb
maxmemory-policy allkeys-lru   # or allkeys-lfu: evict the least frequently used

After switching the policy in the same test, all 20 further SETs succeeded and evicted_keys went to about 20: roughly one old entry thrown out for each new one. What each policy throws out, and why LRU and LFU behave differently under a scan, is covered in LRU Cache. (If the same Redis also holds data that must never be evicted, like queues or rate-limit counters, give caching its own Redis.)

Warming, and the cold start

A freshly started cache is a cold cacheCold cacheA cache that is empty or nearly empty (just started, just flushed), so almost every request misses until it fills up again. A warm cache already holds the popular entries.: every request misses until it fills. After a deploy of the cache, a failover, or a new region, that can be the whole traffic at once. Ways to soften it:

• Warm itCache warmingFilling a cache with the popular entries before real traffic arrives (after a deploy or restart), so the first users don't all miss at once. before sending traffic: replay the most-requested keys (from yesterday's logs or an export of the old cache) through the normal load path.
• Ramp traffic onto a new cache gradually instead of switching all at once.
• Never restart all cache nodes together. Roll them one by one, and give the cache its own replicas so a single failure doesn't empty it.
• Jitter the warmed entries' TTLs (§08), or they'll all expire together one TTL later.

When the cache is load-bearing

Play with the numbers. The question is: if the cache emptied right now, could the database survive?

At 20,000 requests/s and a 99% hit rate, the database sees 200 queries/s, comfortable for one that handles 2,000. Empty the cache and it sees 20,000, ten times its capacity. It doesn't just get slow: queries pile up, time out, get retried, and the cache can't refill because every refill is one of those queries. That's a load-bearing cacheLoad-bearing cacheA cache the system can no longer survive without: if it empties, the misses alone overload the database. Removing it is like removing a wall that holds up the roof., and most successful caches end up as one without anyone deciding it.

Once it's load-bearing, treat the cache like a database:

• Replicate it and plan for failoverFailoverSwitching to a standby (a replica) when the primary server fails, so the system keeps working — automatically or by an operator. (Redis replicas or Cluster; managed services do this for you).
• Protect the database from the misses: coalescing (§07) everywhere, serving stale (§08), and load sheddingLoad sheddingDeliberately rejecting some requests (e.g. HTTP 503) when overloaded, so the rest can still be served quickly. or a circuit breakerCircuit breakerA wrapper around calls to another service that stops calling it for a while after too many failures (“opens”), failing fast instead of piling up waiting requests, then tries again carefully. on the database path, so that a cold cache means "some requests get an error" instead of "everything is down".
• Test it: flush the cache in a staging environment under realistic load and watch what happens. Do it before production does it for you.

Many regions

With servers in several regions, each region gets its own cache: crossing an ocean to reach a cache (100+ ms) defeats the point. But now an invalidation must reach every region's cache, not just the local one.

• A write in Europe deletes the key in Europe's Redis, and must also tell the US and Asia. Usually that means publishing the invalidation to a stream every region consumes, or each region running its own CDC consumer on its local replica.
• Each region's database replica lags differently, so the replica trap from §10 is now per region: a region may refill its cache from a replica that hasn't seen the write yet. Delete again after the lag, or use versions.
• Accept that regions are briefly inconsistent with each other, and design for it (CAP explains why that's unavoidable).

Pitfalls gallery

rdb.Set(ctx, key, data, 0) // 0 = never expires

rdb.Set(ctx, key, data, jittered(10*time.Minute, 0.1)) // always a TTL, jittered (§08)

rdb.Del(ctx, key)
db.Exec(ctx, `UPDATE products SET price_cents = $1 WHERE id = $2`, cents, id)
// ↑ a reader can refill the old value between these two lines

db.Exec(ctx, `UPDATE products SET price_cents = $1 WHERE id = $2`, cents, id)
rdb.Del(ctx, key)

refresh(key, 60, 300, load); // a failure here crashes the process

refresh(key, 60, 300, load).catch((err) => log.warn({ err, key }, "refresh failed"));

Check yourself

Predict the answer before you click. They're covered above: Q1 in §03 and §05, Q2 in §07, Q3 in §09, Q4 in §04, Q5 in §09–10, Q6 in §11.

Cheat sheet