Software AtlasSystem Design Lab
Build an architecture, turn up the traffic, and watch it hold or break. Overloaded parts glow red; add the right piece and it recovers.
Example numbers: an app server handles 5,000 requests/s, a database 5,000 reads/s or 2,000 writes/s. Real systems vary by hardware and workload: measure yours with a load test. The lab scores capacity and cost. The only failure it simulates is the sandbox's 💥 Kill a server. Real designs also keep a spare of each box (N+1N+1 redundancyHaving one more unit than you need (N needed, N+1 running), so the system still copes when any single unit fails. N+2 survives two failures, or one failure during maintenance.), and a replica per shard that takes over if the primary dies.
One server, 100 requests a second
Your side project just launched: about 100 requests a second, mostly people reading pages. Right now there's nothing to answer them. Most requests need an app server (runs your code) and a database (keeps your data).
- Errors under 1%: not met yet100.00%
- p95 latency under 200 ms: not met yet1.0 s
- p95p95 (95th percentile)The value that 95% of measurements are below. A p95 latency of 120 ms means 95 of every 100 requests finished in under 120 ms, and the slowest 5 took longer.: 95 of every 100 requests are faster than this.
- Stars: 1 meets the goals · 2 within 25% of the cheapest answer · 3 the cheapest (10 units). Now: 0
- Errors: not met yet (100.00%)
- p95: not met yet (1.0 s)
Components
Click to add, or drag onto the canvas.Click any box on the canvas to see what it does, how busy it is, and change how many you run.
How the numbers work
Every box has a capacity: how many requests a second it can finish. UtilizationUtilizationThe fraction of time workers are busy. 80% utilization means that, on average, 8 of 10 workers are working at any moment. (how busy it is) is load ÷ capacity. The lab's target is under 60% at peak; the spare room is headroomHeadroomSpare capacity you keep on purpose — machines or workers kept well under full, e.g. 60–80% busy on average — so random bursts can be cleared quickly instead of piling up.. Closer to 100%, requests start waiting in line, and the wait grows faster and faster: with one worker, the average time is the work time ÷ (1 − utilization). Real traffic comes in bursts, so from 85% busy the lab adds a wait that grows until it reaches the 1-second timeout at 100%. Past 100%, almost every answer arrives too late and counts as an error.
The lab models each box as a standard queue with several workers (an M/M/c queue: random arrivals, random job times, c workers) and estimates latency by sending thousands of imaginary requests through your design. The percentilesPercentileThe value below which a given share of measurements fall: p50 is the median; p99 is the value that 99% of requests are at or under. Used for latency because averages hide slow outliers. are approximate: p95 means 95% of requests were faster than this. Failed requests count as the full timeout. Latency includes the trip across the internet.
Read replicas are modelled as paying the full cost of every write they apply (the same 2 ms as the primary); real replicas often apply writes a little more cheaply, so treat the replica numbers as cautious.
💥 Kill a server (in the sandbox) takes one app server down for 10 seconds. Behind a load balancer the others take its share once it fails a health check (a few seconds in real life; instant here), so with a spare (N+1N+1 redundancyHaving one more unit than you need (N needed, N+1 running), so the system still copes when any single unit fails. N+2 survives two failures, or one failure during maintenance.) nobody notices; with just enough servers the rest overload, and with one server every request that needs your servers fails until it's back. With several servers but no load balancer, the lab kills the one users know (the worst case), so the same happens. It's the only failure the lab simulates.
Cost is in made-up units to compare designs (a database machine costs four app servers), not dollars. Read more in why latency explodes near 100% busy and finding the bottleneck.
- App server
- 5,000 req/s
0.8 ms per request, 4 at a time - Database node
- 5,000 reads/s or 2,000 writes/s
0.8 ms per read, 2 ms per write, 4 at a time - Cache node
- 100,000 ops/s
0.5 ms per lookup - Load balancer
- 200,000 req/s per node
0.5 ms - Queue
- 100,000 messages/s
2 ms to accept a write; each worker saves 500/s - CDN
- the static 60% of reads
10 ms away, instead of 40 ms to your servers