The System Design Method

The 30-second version

Whether it's a design review at work or a system design interview, it starts the same way: someone describes a product in two sentences, and you have an empty whiteboard. There are many good designs and no single right one. What people judge is how you got there: did you ask what matters, check the size of the problem, start simple, and notice what breaks?

The method, as a loop

This whole topic teaches one loop. You'll use it on every case study in this area:

Two things about the loop matter more than any single step:

• It's a loop, not a checklist. A number from step 2 can send you back to step 1 ("wait, do we really need to keep snippets forever?"). The bottleneck you find in step 4 might change the data model from step 3.
• Start with the simplest design that could work, often one app server and one database, and only add a box when a number or a requirement demands it. A CacheCacheA small, fast store of answers you already computed or fetched, so the next request for the same thing skips the slow work. The hard part is knowing when a cached answer is out of date. you can't justify is just one more thing that can break.

The running example

To make every step concrete, we'll design one small product all the way through: SnipShare, a site where you paste a piece of text or code and get a short link anyone can open. It's simple enough to hold in your head, but it still needs most of the building blocks: storage, caching, search and a CDNCDNContent Delivery Network: servers spread around the world that keep copies of your files close to users, so they're served from a nearby city instead of your origin server..

Step 1: Pin down the requirements

"Build a snippet-sharing site" sounds clear, until you ask: can people edit a snippet? Do snippets expire? Are there private ones? Is it a hobby project with 1,000 users or a public site with 10 million? Each answer leads to a different design, so asking is part of the work, not a delay before it.

Two kinds of requirement

Beginners tend to list features and stop. But two products with identical features can need completely different designs because of their non-functional requirements. These are the five that come up every time:

"Consistency" here means one specific thing: how stale a read is allowed to be. With copies of the data on several machines, a reader might hit a copy that hasn't received the latest write yet. CAP & PACELC covers this properly; for now, "fresh within a few seconds is fine" is a perfectly good requirement.

Ask the questions that change the design

Not every question is worth asking. A good clarifying question is one whose answer would change a box on the diagram. Click each question to see what it decides:

SnipShare's requirements, written down

After the questions, write a short list everyone can see. This is the version we'll design against for the rest of the topic:

- users can share snippets
- it should be fast
- it should scale
- it should be reliable

Functional
- paste text (up to 1 MB) → get a short link
- anyone with the link can read it
- optional expiry: 1 day / 1 week / never
- search public snippets by keyword

Non-functional
- 10M daily active users, ~10 reads per user per day
- read p99 < 200 ms, write p99 < 1 s
- 99.9% available; saved snippets never lost
- others may see a new snippet up to a few s late

Out of scope (this version)
- accounts and login, comments, edit history

The left version can't be wrong, which is exactly the problem: it gives you nothing to check a design against. The right one gives you numbers to estimate with (next section), a size limit that affects storage, and a scope line that keeps login out of the discussion.

Step 2: Back-of-the-envelope numbers

The numbers decide the design. 10 requests a second fits on a laptop; 100,000 requests a second needs caches, many servers and probably split-up data. You can't tell which one you're building until you multiply a few round numbers together.

The three facts that make it easy

1. Work in powers of ten. Round every number to 1, 2, 3 or 5 times a power of ten, then multiply the leading digits and add up the zeros. 10 million users × 10 reads each = 10⁷ × 10¹ = 10⁸ reads a day. Nobody cares that it's really 97 million.

2. A day has 86,400 seconds, and you round that to 10⁵. So to turn "per day" into "per second", divide by 100,000 (move the decimal point five places left). It's about 15% off, which is fine for this purpose.

3. Byte sizes go up by 1,000 each step. For estimates, use powers of ten: 1 KB = 10³ bytes, 1 MB = 10⁶, 1 GB = 10⁹, 1 TB = 10¹², 1 PB = 10¹⁵. (Strictly, a KiB is 1,024 bytes; at this precision the difference doesn't matter.)

Average vs peak

Traffic is never spread evenly over the day. People use SnipShare during working hours, a link gets shared on a forum, Monday morning is busier than Sunday night. The average QPSQPS (queries per second)How many requests a system handles each second, the basic unit of load in capacity planning. 1 million requests a day is only about 12 QPS on average, but traffic is never spread evenly, so designs plan for the peak. (queries, or requests, per second) tells you how much work there is in total; the peak tells you how many servers you need, because servers have to survive the busiest minute, not the average one.

The usual shortcut is a peak factorPeak factorHow much busier the busiest period is than the daily average. A peak factor of 3 means the busiest hour gets 3× the average requests per second. Servers must survive the peak, not the average.: peak QPS = average QPS × 2 to 10. Consumer products often use 2–3 for normal daily swings; something with bursty events (a ticket sale, a sports final) can be 10× or more.

Worked example: SnipShare

From the requirements: 10 million daily active usersDAU (daily active users)How many different people use the product on a typical day. It's the usual starting number for an estimate, because requests per day = DAU × actions per user per day. (DAU), each reads about 10 snippets a day, and about 1 in 10 users creates one snippet a day. The average snippet is about 10 KB. All numbers are approximate.

Now read what the numbers say, because that's the point of doing them:

• 3,000 reads a second is a modest load. A single well-indexed database can often serve a few thousand simple lookups a second (measure your own hardware before you rely on it). Add a cache and there's lots of room. No need for anything exotic on day one.
• 30 writes a second is tiny. Splitting the database across machines to handle writes would be solving a problem SnipShare doesn't have.
• The system is Read-heavy workloadRead-heavy workloadA system that gets far more reads than writes (a link shortener might see 100 clicks for every new link). Read-heavy systems lean on caches and read replicas; write-heavy ones need different tricks. (100 reads per write), so effort should go into making reads cheap: caching and copies of the data, not faster writes.
• 4 TB a year is the number to watch. Over five years that's 20 TB. And real systems keep 2–3 copies of every byte (replicas and backups, so one dead disk loses nothing): 20 TB × 3 ≈ 60 TB. That's far more than you'd want on database servers, and a hint to keep snippet bodies somewhere cheap and big, like Object storageObject storageA cloud service that stores files ("objects") by key in buckets over HTTP, cheaply and almost without limit (Amazon S3, Google Cloud Storage). Not a disk you mount. (§05), which keeps its own copies for you.
• 30 MB/s is about 240 Mbit/s (×8 to turn bytes into bits). That's fine for a server's network card, and much of it can be served by a CDN anyway.

Your turn

Change the inputs and watch every formula update. Try a 10× bigger product, or make snippets 1 MB instead of 10 KB, and see which number changes the design.

The same estimate in code

Doing it by hand is the skill to build, but a tiny script is handy when the inputs keep changing. This runs as shown and prints the exact (unrounded) numbers. (New to Go? The five-minute primer in Async / Await §02 covers everything used here.)

package main

import "fmt"

func main() {
	const (
		dau           = 10_000_000 // daily active users
		readsPerUser  = 10         // snippets viewed per user per day
		writesPerUser = 0.1        // 1 in 10 users creates one snippet a day
		peakFactor    = 3          // busiest hour vs the daily average
		bytesPerSnip  = 10_000     // 10 KB average snippet
		secondsPerDay = 86_400
	)

	readsPerDay := dau * readsPerUser
	writesPerDay := dau * writesPerUser

	readQPS := float64(readsPerDay) / secondsPerDay // ← per day ÷ seconds per day
	writeQPS := writesPerDay / secondsPerDay
	storagePerYear := writesPerDay * bytesPerSnip * 365 // bytes

	fmt.Printf("reads:   avg %.0f QPS, peak %.0f QPS\n", readQPS, readQPS*peakFactor)
	fmt.Printf("writes:  avg %.0f QPS, peak %.0f QPS\n", writeQPS, writeQPS*peakFactor)
	fmt.Printf("storage: %.2f TB per year\n", storagePerYear/1e12)
	fmt.Printf("egress:  %.0f MB/s at peak\n", readQPS*peakFactor*bytesPerSnip/1e6)
}

reads:   avg 1157 QPS, peak 3472 QPS
writes:  avg 12 QPS, peak 35 QPS
storage: 3.65 TB per year
egress:  35 MB/s at peak

Compare the exact output with the mental math: 1,157 vs "≈ 1,000", 35 MB/s vs "≈ 30". Rounding 86,400 to 100,000 makes estimates about 15% low, which is well within "the right order of magnitude". If an estimate and a script disagree by 10×, one of them has a mistake; if they disagree by 15%, both are fine.

Latency numbers to keep in your head

The previous section estimated how much work there is. This one is about how long each piece of work takes, which is what users feel as LatencyLatencyHow long one request or job takes from the moment it arrives until it's finished — including any time spent waiting in a queue.. A well-known list called "latency numbers every programmer should know" (popularised by Jeff Dean and Peter Norvig in the 2000s) gives the rough cost of common operations. The exact values change with every hardware generation; the ratios have stayed roughly the same, and the ratios are what you design with.

The ladder

Each row is one operation. Toggle between the two scales: the log scale (each grid line is 10× more) lets you see every row; the linear scale shows how they really compare, where everything faster than a network round trip shrinks to an invisible sliver.

The four rungs worth memorising, in round numbers:

(1 ms = 1,000 µs = 1,000,000 ns. The CPU caches topic goes deeper on the fast end of the ladder.)

What the ladder means for a design

A network hop costs more than almost any computation. An app server can do thousands of in-memory operations in the time one call to the database takes. So the first question about a slow request is usually "how many network calls does it make, one after another?", not "is the code efficient?".

You can't beat the speed of light. Light in fiber needs roughly 70–80 ms to cross the Atlantic and come back, no matter how fast your servers are. If SnipShare's servers are in the US and its readers are in Europe, every request pays that before any work happens. The only fix is to put copies of the data closer to the user: a CDNCDNContent Delivery Network: servers spread around the world that keep copies of your files close to users, so they're served from a nearby city instead of your origin server., or servers in more than one regionCloud regionA geographic area where a cloud provider runs its own set of data centers, such as US East (Virginia) or Europe (Ireland). Round trips inside a region take about 1 ms; between continents, 70–250 ms..

Calls in a row add up; calls side by side don't. A request that makes 5 database calls one after another waits 5 round trips. If the calls don't depend on each other, making them at the same time waits about 1. (That's the async patterns topic: Async / Await.)

Spending a latency budget

SnipShare's requirement says "open a snippet in under 200 ms". That 200 ms is a latency budgetLatency budgetThe total time a request is allowed to take (say 200 ms), split among the steps it makes. Each database call, network hop or remote service spends part of the budget.: every step of the request spends part of it. In the widget above, use the + add to request buttons to build a request one call at a time, and watch the budget bar fill.

Try these and see which ones fit:

• One cache read, then one database query, both in the same data center. (Easily: about 1.5 ms of the 200.)
• Fifty database queries one after another, the classic N+1 queries mistake. (Still fits, barely noticeable at 50 ms, but it grows with the page.)
• Two round trips to another continent. (160–300 ms depending on the distance; the widget uses the 150 ms upper end, so 300 ms: over budget, or nearly, before the server does any work.)

Step 3: API and data model first

It's tempting to start with the exciting part, the diagram. But a diagram without an API is a guess: you don't know which requests are hot, which ones must be fast, or what each box actually stores. Writing the APIAPIApplication Programming Interface: the set of functions, types or HTTP endpoints that one piece of software offers for others to call. and the data modelData modelWhat you store and how it's shaped: which tables (or documents, or keys) exist, what fields they have, and how they point to each other. Here it means your design's own schema. first takes five minutes and turns the rest of the design into answering concrete questions.

The API, with numbers on it

Three endpointsEndpointOne operation an API exposes, usually a URL plus a method — e.g. GET /orders/42 or POST /payments. cover SnipShare's functional requirements. The trick is the last column: copy the load from your estimate onto each endpoint. It shows at a glance where the design effort must go.

One endpoint carries about 97% of the traffic (3,000 of about 3,080 requests a second). That single observation tells you most of the design: make GET /snippets/{id} cheap, and the other two can stay simple.

POST /snippets
  body:  { "content": "fmt.Println(…)", "expiresIn": "1w", "public": true }
  reply: 201 Created  { "id": "aZ3kQ9", "url": "https://snip.example/aZ3kQ9" }

GET /snippets/aZ3kQ9
  reply: 200 OK  { "id": "aZ3kQ9", "content": "…", "createdAt": "2026-10-31T09:12:00Z" }
         404 Not Found  if it never existed or has expired

How to name resources, pick status codes and paginate search results is its own topic: API Design. For a system design, the list of calls plus their load is enough.

The data model

Next, what do you store? Write it as tables (or documents), with the fields that matter and how big each record is:

CREATE TABLE snippets (
  id          text PRIMARY KEY,      -- the short id in the link, e.g. 'aZ3kQ9'
  created_at  timestamptz NOT NULL,
  expires_at  timestamptz,           -- NULL = never expires
  is_public   boolean NOT NULL,
  size_bytes  integer NOT NULL,
  body_key    text NOT NULL          -- where the content lives in object storage
);
-- ~100 bytes per row: 1M new rows a day ≈ 100 MB/day ≈ 36 GB/year

Notice what's not in the table: the snippet's content. The estimate said bodies add up to about 4 TB a year, while the metadata rows add up to only about 36 GB. Keeping the big, rarely-changing bodies in Object storageObject storageA cloud service that stores files ("objects") by key in buckets over HTTP, cheaply and almost without limit (Amazon S3, Google Cloud Storage). Not a disk you mount. (cheap, almost unlimited storage for files, addressed by a key) keeps the database small and fast. The body_key column points to the object.

Access patterns choose the database

An Access patternAccess patternOne specific way your app reads or writes data, written as a sentence — “get a customer's last 10 orders, newest first”. Listing them is the first step of choosing and designing a database. is a way the application reads or writes data: "fetch one snippet by id", "search public snippets by word", "delete expired ones". List them, and the storage choice usually becomes obvious:

Lookups by key at modest volume mean almost any database works, so pick the one the team already runs well (here, Postgres). "Which database?" is rarely the interesting question in a design; the access patterns are. SQL vs NoSQL walks through matching patterns to databases.

The building blocks

Each block below has its own deep-dive. This page is the index: what it is in one line, the symptom that tells you to reach for it, and what it costs you. Every block fixes one problem and adds at least one new one, usually another thing that can fail or go stale.

Symptom → block

When the numbers or a failure point at a problem, this is the usual first move. It's a starting point, not a rule:

SnipShare with every block

Here's where SnipShare could end up: the read path runs top to bottom (browser → CDN → load balancer → app → cache, then the database and object storage on a miss), and saving a snippet also drops a job on the queue so a worker can add it to the search index. It's not where it starts: §07 grows it from one server, one box at a time, and you'll see that at SnipShare's real numbers, several of these boxes aren't needed yet.

Step 4: Start simple, find the bottleneck, change one thing

Start with the simplest thing that works

The first design should be embarrassingly simple: one app server and one database. For SnipShare's real numbers (about 3,000 reads a second at peak) it isn't quite enough, as you'll see below, but it's the right starting point, because every box you add later will then have a reason written next to it.

Starting big has real costs. Each extra box is something to deploy, monitor, pay for and debug at 3 a.m., and the design becomes harder to change. "We might need it later" is how teams end up running a cluster of message brokers for 20 requests a second. (The principle has a name, YAGNIYAGNI“You Aren't Gonna Need It”: don't build features or flexibility until a real need shows up. Predicted needs are usually wrong.: you aren't gonna need it.)

Draw the request path

The request pathRequest pathEvery box a single request passes through, in order, from the user's device to the data and back: for example browser → CDN → load balancer → app server → cache → database. is every box one request passes through. For the simple design, reading a snippet goes: browser → app server → database → app server → browser. Now put the load from §03 on each box and compare it with what the box can handle. The box whose load is closest to its limit is the BottleneckBottleneckThe one resource that runs out first under load — CPU, a connection pool, a lock, a slow dependency. It sets the maximum throughput for the whole system., and it sets the maximum for the whole system, no matter how much room the others have.

A useful way to say "how close to its limit" is UtilizationUtilizationThe fraction of time workers are busy. 80% utilization means that, on average, 8 of 10 workers are working at any moment.: load ÷ capacity. A server that can handle 5,000 requests a second and gets 4,000 is 80% busy. As a box gets close to fully busy, requests start queueing and latency shoots up long before it hits 100%; Worker Pools shows why. So this topic uses a simple target:

The loop, by hand

1. Draw the request path with the load on each box.
2. Find the box with the highest utilization.
3. Change one thing that relieves that box: a component from §06.
4. Recompute the numbers. Repeat until every box is under the target at peak and nothing is a single point of failure.

Changing one thing at a time matters for the same reason it matters when debugging: if you add a cache, three replicas and a CDN at once, you can't tell which one helped, and you're paying for all three.

Try it: grow SnipShare

The simulator starts as the simplest design at 100 QPS. Drag the traffic up. Each time a box turns red, add one component to fix it, and watch where the red moves next. Then switch to 10 : 1 reads (a more write-heavy product) and try again.

Things to notice while you play:

• The fix depends on the cause. A database overloaded by repeated reads wants a CacheCacheA small, fast store of answers you already computed or fetched, so the next request for the same thing skips the slow work. The hard part is knowing when a cached answer is out of date.. A database overloaded by writes doesn't care about the cache at all.
• Replicas don't add write capacity. Every ReplicaReplicaA full copy of a database kept on another machine and updated as the original changes, for reading load and for taking over if the original dies. applies every write the primary does, so with heavy writes the "per node" write load never drops. Only splitting the data into shardsShardOne piece of a split-up dataset, together with the database server that stores it. “Tenant 42 is on shard 3” means its rows live on that server. divides writes. Sharding explains how, and why it's usually the last thing you reach for.
• You can't add a second app server without a load balancer. Users need one address to connect to; the load balancer is that address, and it spreads requests across the servers behind it.
• The cost line goes up with every box. Watch "machines you run". At 3,000 QPS, a sensible design is a handful of machines. If you're running more, the simulator tells you which ones the numbers don't need.

SnipShare at its real numbers

Back to the estimate: about 3,000 read QPS and 30 write QPS at peak, read-heavy. Running the loop by hand (with the simulator's capacities, which are made up):

That's the honest answer for today's SnipShare: two app servers, a load balancer, a cache and one database (plus object storage for the bodies, from §05). A read replica will come back in §09, but for failure, not for load.

Trade-offs: every choice costs something

Every box from §06 fixed a problem and added one. A cache made reads fast and made them possibly stale. A second app server added capacity and required a load balancer. This section covers the trade-offs that come up in almost every design, and how to talk about them.

Consistency vs availability vs latency

As soon as data lives in more than one place (replicas, caches, a second region), a write takes time to reach every copy. While it travels, the copies disagree. You have to pick what happens then:

• Consistency first: every reader sees the latest write, or gets an error. To guarantee that, writes must wait until enough copies have confirmed, which costs latency, and when copies can't reach each other, some requests must be refused, which costs availability.
• Availability first: every request gets a fast answer, from whatever copy is nearby, even if that copy is a little behind. That's Eventual consistencyEventual consistencyA promise that if writes stop, every copy or derived view of the data (replicas, caches, read models, other services) will end up agreeing — but for a while, different readers may see different (stale) versions.: the copies agree eventually, once the update arrives.

The CAP theoremCAP theoremA proven result (Gilbert & Lynch, 2002, from a 2000 talk by Eric Brewer): when the network between replicas splits, a system can't both answer every request and keep every answer up to date. It must give up one of the two until the network heals. is the formal version: when the network between copies breaks (a Network partitionNetwork partitionWhen the network splits so that some nodes can't reach others, even though all of them are still running. Each side may think the other side is dead.), a system must choose between staying correct and staying available. PACELCPACELCAn extension of CAP (Daniel Abadi, 2010): if there's a Partition, choose Availability or Consistency; Else, in normal operation, choose lower Latency or Consistency. adds the everyday half: even when the network is fine, you trade latency for consistency. Both are taught properly in CAP & PACELC. Try the choice yourself:

Choose per piece of data, not per system

The most useful insight about this trade-off is that you don't make it once. Different data in the same product needs different answers:

SnipShare's requirement "others may see a new snippet up to a few seconds late" is exactly a decision to favour availability and latency for snippet content. Writing that sentence down in §02 is what lets you use replicas and caches freely now.

Cost vs complexity

Every box costs twice: once in money (machines, managed services, network) and once in complexity: more to deploy, monitor and understand, more ways to fail. The second cost is usually bigger, and it never shows up on the bill.

6 microservices, Kafka, Cassandra,
Kubernetes across 3 regions
→ peak load: 30 writes/s, 3,000 reads/s
→ a team of 4 spends most of its time
  operating infrastructure

2 app servers + load balancer,
managed Postgres + standby replica,
managed Redis, object storage
(+ a CDN once far-away users need it, §04)
→ same load, lots of headroom
→ the team ships features

The simple design isn't the "cheap" option; it's the one that matches the numbers. It also leaves a clear upgrade path: each next box has a number attached that tells you when to add it.

Build vs buy

Build vs buyBuild vs buyThe choice between writing and running a component yourself and using an existing product or managed cloud service. Building gives control; buying gives speed and someone else to wake up at night, for a price. is the choice between running a component yourself and using a product or managed service. For SnipShare, "buy" means a managed database, cache, load balancer and object storage, which matches the "small team" requirement from §02.

The hidden cost of buying is lock-in: moving off a managed service later can be a large project. The hidden cost of building is the on-call rotation. Cloud Basics goes deeper on managed vs self-hosted.

How to state a trade-off

A trade-off said well has four parts: the choice, what it buys, what it costs, and why the cost is acceptable here. Compare:

• ❌ "We'll use eventual consistency because it scales better."
• ✅ "Snippet reads are served from the cache and the CDN, so a new snippet can take up to a few seconds to appear for other readers. That's allowed by the requirements, and it keeps the database free for writes. There are no accounts, so after saving, the author's browser gets a short-lived cookie; for a minute, requests with that cookie read from the database directly, so the author never sees their own snippet missing."

The second one can be checked: a reviewer can disagree with the requirement or with the "minute", instead of with a slogan. (That last trick gives the author Read-your-writesRead-your-writesA guarantee that after you change something, your own later reads show the change. Other users may still see the old value for a moment.: Replication shows how to route those reads.)

Step 5: What if each box dies?

So far the design handles the load. But machines crash, disks fill up, networks drop packets and deploys go wrong. A design that's only correct when everything works is half a design. The method is mechanical, which is what makes it useful: point at each box in turn and ask "what if this dies?"

Three questions per box

1. What do users see? Nothing? Slower pages? One feature missing? The whole site down?
2. How does it recover? Automatically (a load balancer's health check skipping a dead server, or a FailoverFailoverSwitching to a standby (a replica) when the primary server fails, so the system keeps working — automatically or by an operator. to a standby copy), or does a person have to do something?
3. How would we know? What metric or alert tells us before users do?

Two answers are red flags. "The whole site is down" for a box that has no spare means it's a single point of failureSingle point of failure (SPOF)One component that, if it breaks, takes the whole system down with it — because everything depends on it and nothing can take its place.. And "we'd find out from users" means you're missing monitoring.

Try it

This is SnipShare's design with every block. Click (or tap) a box to kill it, and look at what happens to each of the three endpoints. Then kill two at once: the cache first, then the cache and the primary database.

Patterns that came up

Redundancy: run one more than you need. The app tier survives a dead server because there are two behind a Load balancerLoad balancerA component that spreads incoming requests or connections across several servers, so no single one is overloaded and a dead one can be skipped. that runs health checksHealth checkA small automatic request (e.g. GET /healthz every 5 s) a load balancer or orchestrator sends to each server to decide whether it should still receive traffic.. Each runs at about 30% at peak (§07); if one dies, the other carries everything at about 60%, right at our target. The rule of thumb is N+1 redundancyN+1 redundancyHaving one more unit than you need (N needed, N+1 running), so the system still copes when any single unit fails. N+2 survives two failures, or one failure during maintenance.: enough capacity for the peak, plus one spare, so a failure (or a deploy) during the peak doesn't overload the rest. That's also why §07's "one server at 60%" wasn't good enough.

Failover: a standby that can take over. The database primary can't simply be duplicated, because two primaries accepting writes would disagree. Instead a ReplicaReplicaA full copy of a database kept on another machine and updated as the original changes, for reading load and for taking over if the original dies. follows it and gets promoted when it dies. That's the real reason SnipShare has a replica: for failure, not for load. One catch: replication is usually an asynchronous writeAsynchronous writeA write the system acknowledges (“saved!”) before every copy or follow-up step is done; the rest finishes in the background. Fast for the user, but other readers may briefly see the old data., so the primary says "saved" before the replica has the data, and the last few writes before a crash can be missing after promotion. For "never lose a saved snippet", the primary must wait for the replica to confirm each write. How promotion works and what can be lost is in Replication.

Know what your CDN is hiding. If the CDN fails completely, every read it used to answer lands on your own servers, which CDN people call the origin. Plan for the origin to survive that, or for a second CDN.

Watch for load-bearing caches. Losing the cache didn't break anything by itself, but it multiplied the database's load by 10. If the database can't survive that, the cache has quietly become something the system can't live without. Either size the database for a cold cache, or protect it, for example with request coalescing: when many requests miss on the same key at once, only one goes to the database and the rest wait for its answer. Caching explains it.

Graceful degradation: let optional features fail alone. Search being down shouldn't stop people reading snippets. Keep non-essential features off the critical path: call them with short timeouts, and show "unavailable right now" instead of an error page. Graceful degradationGraceful degradationDesigning a system so that when a part fails, it keeps working with less — showing a default list instead of personalized recommendations — instead of failing completely. is a design decision you make in advance, not something that happens by luck.

Decouple slow work with a queue. Indexing a snippet for search happens in a background worker fed by a queue. When the worker dies, the queue holds the backlog and nothing else breaks. The cost: you must decide what happens if the queue itself is down, and handle jobs that run twice (Idempotency covers that). A common answer is the outbox pattern: save the snippet and an "index this" row in the same database transaction, and let a small relay move rows from that table to the queue. Saving never depends on the queue being up, and no job is lost (Sagas: outbox and inbox).

Correlated failures

The toggles above kill one box at a time, but real outages often take several at once: a whole data center (an Availability zoneAvailability zone (AZ)One of a few separate data centers inside a cloud region (e.g. us-east-1a, 1b, 1c), with its own power and network, so one can fail without taking down the others.) loses power, a bad config is deployed everywhere, a shared dependency like DNS breaks. Two replicas on the same rack aren't really two. That's why production designs spread copies across zones, and roll out changes gradually. Reliability & SRE covers planning for losing a whole zone.

SLOs as requirements, and capacity with headroom

§02 wrote "read p99 < 200 ms" and "99.9% available" as requirements. At a staff level, those sentences have to become things you can measure, alert on, and use to decide what to build next. This section connects the design to the operational topics: Observability and Reliability & SRE.

From requirement to SLO

An SLISLI (service level indicator)A measured number that captures what users experience, like “fraction of requests answered successfully in under 300 ms”. is a measured number that reflects what users experience. An SLOSLO (service level objective)A reliability target you measure, like “99.9% of requests succeed in under 300 ms over 30 days”. The measured number (here, the share of fast successful requests) is the SLI; the 0.1% you're allowed to miss is the error budget. is the target for it over a time window. The gap between 100% and the target is the Error budgetError budgetHow much failure an SLO allows: at 99.9% over 30 days, about 43 minutes of bad requests. While budget remains you can ship risky changes; when it's spent you slow down and fix reliability.: how much failure you're allowed before reliability work takes priority over features.

Notice the last one: it measures the queue + indexer path from §09. If you only measure what the app server does, an indexer that has been dead for a day looks perfectly healthy. An SLO per user-visible promise catches failures in the background parts too.

What the nines mean

"Nines"“Nines”Shorthand for availability targets: three nines = 99.9% (about 43 minutes of downtime a month), four nines = 99.99% (about 4.3 minutes). Each extra nine allows 10× less downtime and costs much more. are shorthand for availability targets. Over a 30-day month (43,200 minutes), each one allows this much failure:

Each nine allows 10× less downtime and typically costs far more than 10× the effort: automatic failover everywhere, multiple zones or regions, very careful deploys. That's why "how many nines?" is a requirements question with a cost attached, not a technical detail.

Availability multiplies along the request path

If a request needs three boxes in a row and each is available 99.95% of the time or better, the request succeeds only when all three are up, so you multiply. A worked example with round, made-up numbers:

• Load balancer 99.99% × app tier 99.95% × database 99.95%
• = 0.9999 × 0.9995 × 0.9995
• ≈ 0.9989, which is 99.89%: below a 99.9% target, even though every part beats it.

Every dependency on the critical path lowers the ceiling. That's another argument for the simple design: fewer boxes in a row means fewer numbers to multiply. It also explains graceful degradation from §09: search isn't on the read path, so its downtime doesn't count against reading.

Redundancy works the other way. Two copies that fail independently, each up 99% of the time, are both down only 0.01 × 0.01 = 0.01% of the time: 99.99% together. The catch is the word "independently": two copies in the same zone, running the same buggy config, fail together. That's why correlated failure (§09) is the thing to design against.

Capacity planning with headroom

Capacity planningCapacity planningWorking out how many servers (or how much database, cache, bandwidth) you need for the traffic you expect, plus a safety margin — using numbers measured in load tests. asks: will we have enough of everything for the busiest moment coming up, even if something breaks? Here's the app tier for SnipShare, step by step. Unlike the simulator, these inputs should be measured, with a load test, not guessed:

Why 15 here when §07 ended with 2? §07 used the simulator's made-up 5,000 QPS per server and only today's load. Here a real load test says 1,000 QPS per server, we keep 40% spare, plan for next year, and survive losing a zone. Same method, honest inputs, very different answer: that's why capacity planning uses measurements, not guesses.

1. Today's peak: about 3,500 QPS (the exact figure from the §03 script).
2. Growth: the product team expects +50% over the next year → forecast peak 3,500 × 1.5 = 5,250 QPS.
3. Per-server capacity: a load test shows one app server handles about 1,000 QPS before p99 latency breaks the SLO.
4. Target utilization: plan to run at most 60% busy at peak, which leaves HeadroomHeadroomSpare capacity you keep on purpose — e.g. workers that are only 80% busy on average — so random bursts can be cleared quickly instead of piling up. for spikes and slow moments → each server counts as 600 QPS.
5. Survive losing a zone: servers are spread across 3 zones; if one dies, the other two (2/3 of the fleet) must carry the whole peak → 5,250 ÷ (600 × 2/3) ≈ 13.1 → round up to 5 per zone = 15 servers.

package main

import (
	"fmt"
	"math"
)

// serversNeeded: enough servers that, even with one zone down,
// the rest run at or below targetUtil at the forecast peak.
func serversNeeded(peakQPS, perServer, targetUtil float64, zones int) int {
	healthy := float64(zones-1) / float64(zones) // share left after losing one zone
	n := peakQPS / (perServer * targetUtil * healthy)
	perZone := math.Ceil(n / float64(zones)) // same count in every zone
	return int(perZone) * zones
}

func main() {
	forecast := 3500 * 1.5 // today's peak × expected growth over the next year
	fmt.Println("forecast peak:", forecast, "QPS")
	fmt.Println("servers:", serversNeeded(forecast, 1000, 0.6, 3))
}

forecast peak: 5250 QPS
servers: 15

Fifteen servers, when today's peak could be carried by 4 servers running at nearly 90% busy, can look wasteful. It isn't: it's the price of "a whole zone can die on our busiest day of the year and users don't notice". The same reasoning applies to everything else with a limit: database connections, cache memory, disk, queue throughput, and cloud quotas. Reliability & SRE covers forecasting and the limits that aren't servers.

Step 6: Write it down in a design doc

At work, the output of the method isn't a whiteboard photo: it's a design docDesign docA short written proposal for how a system will be built, shared before building it: the problem, requirements, numbers, the proposed design, alternatives considered, and risks. Reviewers comment on it before code exists. (some teams call it an RFC, "request for comments", because it's circulated for others to comment on). It records the requirements, the numbers, the design and, most importantly, the reasoning, so reviewers can disagree with a specific assumption instead of with the whole thing. Six months later it also answers "why on earth did we build it this way?".

A one-page template

Every section maps to a step of the method. Short is better: a reviewer should get the main idea in five minutes.

# SnipShare v1: paste a snippet, share a link
Author · Reviewers · Status (draft / in review / approved) · Date

## Context and goals        ← why now, what problem, for whom
## Requirements             ← §02: functional, non-functional (as SLOs), out of scope
## Estimates                ← §03: QPS avg/peak, storage/year, bandwidth, read:write
## API and data model       ← §05: endpoints with load, schema, access patterns
## Proposed design          ← §06–07: diagram, request path, why each box exists
## Alternatives considered  ← what else you could do, and why not
## Failure modes            ← §09: what if each box dies; what we monitor
## Rollout and migration    ← how it gets to production safely; how to undo it
## Risks and open questions ← what you don't know yet, and how you'll find out

SnipShare v1, filled in

Here's the whole method's output for SnipShare in one place: one or two lines per heading, pulling together §02–§10.

# SnipShare v1: paste a snippet, share a link
Context    Developers want to share code without accounts. Goal: launch v1 for ~10M daily users.
Reqs       Paste ≤ 1 MB → short link; read by link; expiry; keyword search on public snippets.
           Read p99 < 200 ms, write p99 < 1 s, 99.9% available, saved snippets never lost,
           others may see new snippets a few s late. Out of scope: accounts, comments, edits.
Estimates  ≈ 3,000 read QPS + 30 write QPS at peak (100:1, read-heavy); ≈ 4 TB bodies/yr
           (≈ 60 TB over 5 yr with 3 copies); ≈ 36 GB metadata/yr; ≈ 30 MB/s out at peak.
API/data   POST /snippets, GET /snippets/{id} (97% of traffic), GET /search.
           Postgres row per snippet (~100 B) + body in object storage under body_key.
Design     LB → app servers → Redis cache (90% hits) → Postgres primary + standby replica;
           bodies in object storage; on save, an outbox row → queue → indexer → search index.
           Every box under 60% at peak. App tier: 2 servers on paper (§07); sized from a
           load test for launch: ≈ 15 across 3 zones for next year's peak (§10).
Alternatives  Bodies in Postgres (backups too big); DynamoDB (no need to switch);
           replicas instead of a cache (a cache is cheaper for unchanging snippets).
Failures   App server or a whole zone dies → the rest stay under target (N+1). Primary dies → promote replica
           (writes wait for the replica, so nothing saved is lost). Cache dies → DB must
           survive 10× reads. Search/indexer down → search degraded, core unaffected.
SLOs       99% of reads < 200 ms; 99.9% without server errors; 99% searchable within 5 min.
Rollout    Launch behind a feature flag; load test to 2× forecast peak first.
Risks      Real per-server capacity unknown until load tested (§10 sizing depends on it);
           short-id collisions across servers (see the URL shortener case study).

The section reviewers read first

Experienced reviewers often jump straight to Alternatives considered. A design with no alternatives suggests the author picked the first idea that came to mind. Two or three real options, each with one line on why not, show the trade-offs were weighed:

"Do nothing" or "the simplest possible version" belongs in the list too. Sometimes it wins.

Running a design review

• Send the doc ahead. Reviewers comment in writing first; the meeting is for the disagreements that remain.
• Ask reviewers for the questions that hurt. "Which number would make this design wrong?" "What happens on the busiest day if the primary dies?" "What's the plan to undo this?"
• Record the decisions in the doc, including the ones you rejected. That's the document's long-term value.

The second-system trap

The second-system effectSecond-system effectThe tendency to over-design the replacement for a system that worked: every feature and fix you wished the first one had goes in at once, so the new one is late, complex and fragile. Named by Fred Brooks in “The Mythical Man-Month” (1975). is the urge, when replacing a system that worked, to put every improvement you ever wanted into the replacement at once. Fred Brooks described it in The Mythical Man-Month (1975), and it still happens: the "v2 rewrite" that is general enough for every future need, takes three times as long as planned, and ships with more problems than v1 had.

Warning signs in a design doc:

• Requirements that no user asked for ("pluggable storage backends", "support for any future content type").
• Capacity planned for 100× today's numbers with no forecast behind it.
• More new components than problems listed in "Context and goals".
• No step-by-step migration: the new system replaces the old one in a single big switch.

The fix is the same as in §07: solve the problems the numbers and requirements show, and leave a clear path for the rest. Migrate in small steps, keeping the old and new systems side by side, so each step can be checked and undone. (The strangler fig pattern is the classic way to do this.)

Common mistakes

Check yourself

Work each answer out on paper before you click: a quick estimate is the skill being tested. Q1, Q2 and Q6 practise §03; Q3 is §06–07; Q4 is §04; Q5 is §09; Q7 is §02.

Cheat sheet