Load Testing

The 30-second version

Your other tests ask "does it give the right answer?" A load test asks a different question: "does it still give the right answer, quickly, when 500 people ask at once?" A function that takes 20 ms for one user can take 2 seconds when hundreds of users are waiting in line for the same database.

Try it

Here's a tiny simulated server. It has 8 "slots" (think: 8 database connections), and every request holds one slot for about 20 ms. Drag the slider to send more requests per second, and watch 20 seconds play out.

Try 200/s, then 380/s, then 450/s. Up to about 380 per second nothing changes — every request takes ~20 ms. Past ~400/s, the server can't finish requests as fast as they arrive, a line forms, and the wait grows every second the traffic continues. That sudden bend is called the knee, and finding it is the main job of a load test (§09).

What this topic covers

1. What to measure — four numbers, and why averages lie (§02).
2. Which test to run — smoke, load, stress, spike, soak (§03).
3. Hands-on with k6k6A free, open-source load-testing tool from Grafana Labs. You write the test in JavaScript; k6 itself is a single program written in Go that runs it and measures every request. — a real Go server, a real k6 script, real output, and pass/fail rules for CI (§04–§06).
4. Generating traffic correctly — the difference between "50 users clicking" and "300 requests per second", and a measurement trap that makes slow systems look fast (§07–§08).
5. Using the results — find the knee, then turn it into "how many servers do we need?" (§09–§10), and how to test somewhere that resembles production (§11).

What to measure: four numbers

Think of a coffee shop at the morning rush. You'd want to know: how many coffees per minute come out (throughput), how long people wait (latency), how many orders come out wrong (errors), and whether the baristas are at 100% with a line out the door (saturation). One number alone never tells the whole story.

1. Throughput

ThroughputThroughputHow many jobs get finished per second. For a worker pool, the maximum is roughly workers ÷ time per job. is how many requests the service finishes per second, written as requests per second (RPS)Requests per second (RPS)How many requests arrive at (or are finished by) a server each second. The usual unit for throughput in web services.. The key habit: compare it with what you asked for. If the test sends 500/s and only 388/s complete, the other 112 per second are piling up somewhere — a line is forming.

2. Latency — as percentiles, never an average

LatencyLatencyHow long one request or job takes from the moment it arrives until it's finished — including any time spent waiting in a queue. is how long one request takes, from sending it until the full reply arrives. A load test produces thousands of latencies, and you need to summarize them. Don't use the average. A worked example with 100 requests:

• 98 take 20 ms, 2 get stuck and take 2,000 ms.
• Average = (98 × 20 + 2 × 2,000) ÷ 100 = 59.6 ms. Nobody actually waited 59.6 ms.
• p50 (the median — sort all 100, take the middle) = 20 ms: what a typical request feels.
• p99 (the time 99% of requests stay under) = 2,000 ms: what the unlucky ones feel.

A PercentilePercentileThe value below which a given share of measurements fall: p50 is the median; p99 is the value that 99% of requests are at or under. Used for latency because averages hide slow outliers. says "X% of requests were this fast or faster". Load-test reports usually show p50, p95 and p99. The slow end is called the tailTail latencyThe latency of the slowest few requests — usually reported as p99 or p99.9. It's what your unluckiest users feel, and it grows much faster than the average under load., and it grows first when a system gets busy. For the full story — histograms, why you can never average two p99s — see Observability: percentiles.

3. Error rate

The Error rateError rateThe share of requests that failed — a network error, a timeout or an error status like 500. 3 failures out of 1,000 requests = 0.3%. is the share of requests that failed: timeouts, refused connections, 500 responses. Read it together with latency, because errors are often fast. Returning "500 Internal Server Error" takes almost no work, so a service that fails half its requests can show lovely latency numbers.

4. Saturation

SaturationSaturationHow much extra work is waiting because a resource is fully busy — e.g. requests queued for a database connection. Utilization near 100% plus a growing queue means saturated. is how full your busiest resource is — CPU, memory, database connections — and whether work is waiting for it. It's the early warning: users feel nothing yet, but when the connection pool is busy 97% of the time, a little more traffic will tip it over. Your load-testing tool can't see this; you read it from the server's own metrics during the test (see Observability: metrics).

Practice

Smoke, load, stress, spike, soak

Back to the bridge. Engineers run several different checks: one truck to see the sensors work, a normal rush hour, an overloaded convoy, a sudden stampede, and a month of ordinary traffic to look for slow fatigue cracks. Each answers a different question, and you wouldn't run the last one first.

export const options = {
  stages: [
    { duration: "5m", target: 100 },  // ramp up to a normal day
    { duration: "30m", target: 100 }, // stay there
    { duration: "5m", target: 0 },    // ramp down
  ],
};

What each one is for

• A Smoke testSmoke test (load)A tiny load test — often 1 virtual user for a minute — to check the script works and the system answers at all, before running anything bigger. is the "does it even work?" check: one or two virtual users for a minute. It catches a broken script or a dead endpoint in seconds, so it runs first — and it's cheap enough to run on every pull request.
• A load test (sometimes "average-load test") replays a normal busy day. It answers "do we meet our targets today?"
• A Stress testStress testA load test that pushes traffic beyond what you normally expect, to see what breaks first and whether the system fails gracefully. goes above normal. It answers "what breaks first, and how?" A good system slows down or rejects extra work politely (see load shedding); a bad one crashes and takes everything with it.
• A Spike testSpike testA load test where traffic jumps suddenly (say 10× in seconds), holds, and drops back — to check the system survives bursts and recovers afterwards. jumps suddenly. It checks the system survives the burst and — just as important — recovers when traffic drops back. Auto-scalingAutoscalingAutomatically adding instances when a metric (CPU, requests per second, latency) is high and removing them when it's low. that needs 3 minutes to add servers won't help with a 30-second spike.
• A Soak testSoak testA load test at normal traffic for many hours, to catch problems that only appear slowly: memory leaks, connection leaks, growing queues, filling disks. runs normal traffic for hours. Some bugs only appear slowly: a Memory leakMemory leakMemory that your program no longer needs but can't free, because something still holds a reference to it. In garbage-collected languages, a forgotten listener or map entry is the usual culprit. that grows 10 MB per hour, database connections that are never returned, a log file filling the disk.
• A Breakpoint testBreakpoint testA load test that raises traffic steadily until the system stops meeting its targets, to find the maximum it can handle. keeps raising traffic until a target is broken, to find the maximum. That's how you find the knee in §09.

Ramp up, don't jump

Notice that most shapes ramp up gradually. Starting at full traffic in the first second gives misleading results: caches are empty, connections aren't open yet, and auto-scaling hasn't reacted. The first minutes of any test are a Warm-upWarm-upThe first minutes of a test, when caches are still empty, connections are being opened and (in some runtimes) code is still being optimized. Results from this period are usually slower than steady state.; judge the system on the steady part after it.

Your first k6 test against a Go server

The target: a server with a bottleneck

To learn anything from a load test, the server needs a realistic limit. This one pretends every request needs one of 8 database connections (a Connection poolConnection poolA set of open database (or HTTP) connections that your program keeps and reuses instead of opening a new one per request. Pools usually have a size limit; when every connection is busy, the next caller waits for one to be returned.) for 15–25 ms. That pool is its BottleneckBottleneckThe one resource that runs out first under load — CPU, a connection pool, a lock, a slow dependency. It sets the maximum throughput for the whole system.: the thing that runs out first.

package main

import (
	"fmt"
	"log"
	"math/rand/v2"
	"net/http"
	"time"
)

// Only 8 requests can be "at the database" at once — like a pool of 8 connections.
var pool = make(chan struct{}, 8)

func products(w http.ResponseWriter, r *http.Request) {
	pool <- struct{}{}        // take a slot (waits while all 8 are busy)
	defer func() { <-pool }() // give the slot back when the handler returns

	work := 15*time.Millisecond + rand.N(10*time.Millisecond) // 15–25 ms of "work"
	time.Sleep(work)

	w.Header().Set("Content-Type", "application/json")
	fmt.Fprintln(w, `{"products":[]}`)
}

func main() {
	http.HandleFunc("GET /api/products", products)
	log.Println("listening on :8080")
	log.Fatal(http.ListenAndServe(":8080", nil))
}

In the Node version, acquire and release do the same job by hand: a counter of busy slots plus a list of requests waiting in line. Node handles each request on its single thread, but await sleep(…) lets it serve others while one waits — see Async / Await.

With 8 slots and ~20 ms per request, the server's ceiling should be about 8 ÷ 0.020 s = 400 requests per second. Let's see whether a load test agrees.

Install k6 and write the test

k6k6A free, open-source load-testing tool from Grafana Labs. You write the test in JavaScript; k6 itself is a single program written in Go that runs it and measures every request. is a single program (written in Go) that runs test scripts written in JavaScript. Install it from your package manager (brew install k6, or the packages listed on the k6 website), or run it with Docker without installing anything:

go run .                                  # start the server (or: node server.mjs)

# in another terminal:
k6 run load.js
# …or with Docker (--network host lets the container reach localhost:8080 on Linux):
docker run --rm -i --network host grafana/k6 run - < load.js
# On Mac/Windows Docker Desktop: drop --network host and use
# http://host.docker.internal:8080 instead of http://localhost:8080 in the script

import http from "k6/http";
import { check, sleep } from "k6";

export const options = {
  vus: 10,          // 10 virtual users…
  duration: "30s",  // …each looping for 30 seconds
};

export default function () {
  const res = http.get("http://localhost:8080/api/products");
  check(res, { "status is 200": (r) => r.status === 200 });
  sleep(1); // think time: a real user pauses between clicks
}

What one virtual user actually does

1import http from "k6/http";
2import { check, sleep } from "k6";
3 
4export const options = {
5  vus: 10,          // 10 virtual users…
6  duration: "30s",  // …each looping for 30 seconds
7};
8 
9export default function () {
10  const res = http.get("http://localhost:8080/api/products");
11  check(res, { "status is 200": (r) => r.status === 200 });
12  sleep(1); // think time: a real user pauses between clicks
13}

The important arithmetic from the last steps: each user does one request every ~1.02 s, so 10 users produce only about 10 requests per second — far below the server's ~400/s. The number of virtual users is not the number of requests per second. §07 shows how to ask for an exact request rate instead.

Reading the results

Here's what the §04 run printed at the end, trimmed (… marks cut lines) (measured on this machine; your numbers will differ a little):

     scenarios: (100.00%) 1 scenario, 10 max VUs, 1m0s max duration (incl. graceful stop):
              * default: 10 looping VUs for 30s (gracefulStop: 30s)

  █ TOTAL RESULTS

    checks_total.......: 300     9.767492/s
    checks_succeeded...: 100.00% 300 out of 300
    checks_failed......: 0.00%   0 out of 300

    ✓ status is 200

    HTTP
    http_req_duration..............: avg=21.9ms min=15.74ms med=21.14ms max=66.91ms p(90)=25.31ms p(95)=26.2ms
      { expected_response:true }...: avg=21.9ms min=15.74ms med=21.14ms max=66.91ms p(90)=25.31ms p(95)=26.2ms
    http_req_failed................: 0.00% 0 out of 300
    http_reqs......................: 300   9.767492/s
    …

The header line gracefulStop: 30s is the graceful stopGraceful stop (k6)How long k6 lets iterations that are still running finish after a test or stage ends, before cutting them off. 30 seconds by default.: after 30 s of testing, k6 gives iterations still in flight up to 30 more seconds to finish. Click through the lines below to see what each one means:

The verdict for this run

Latency is excellent — p95 of 26 ms is basically the server's 15–25 ms of work plus a little overhead. No errors, all checks passed. But the test proves almost nothing, because the server only saw ~10 requests per second, about 2.5% of what it can handle. The bridge held one car.

Why 10 users made only 10 requests per second

Each virtual user waits for its reply, then sleeps 1 s, then sends again. So one user's loop takes about 21 ms + 1,000 ms ≈ 1.02 s, and:

requests per second ≈ virtual users ÷ (latency + think time) = 10 ÷ 1.02 s ≈ 9.8/s

This is Little's LawLittle's LawL = λ × W: the average number of things in a system equals the rate they arrive times how long each one stays. 100 requests/s × 0.2 s each = 20 in progress at any moment. in disguise (explained in Worker pools: sizing). Little's Law says L = λ × W: things in the system (L) = arrival rate (λ) × time each one stays (W). Here L = 10 virtual users, W = 1.02 s per loop, so λ = L ÷ W = 10 ÷ 1.02 ≈ 9.8 requests per second. It has an important consequence: with a fixed number of users, if the server gets slower, the users automatically send less. Each user is stuck waiting longer, so fewer requests go out. Keep that in mind — §07 and §08 are about why it's a problem.

To push harder you could remove the sleep or add users. Here's what happened with no sleep and more users (15 s each, measured on this machine):

Up to 8 users, every user gets its own slot: throughput doubles, latency stays flat. Beyond 8, throughput stops growing at the ~388/s ceiling (a bit under the theoretical 400 because of HTTP overhead), and latency grows instead: 64 users ÷ 388 per second ≈ 165 ms, exactly what p50 shows. Extra users just stand in line.

Useful flags

# show p99 too (k6 shows avg, min, med, max, p(90), p(95) by default)
k6 run --summary-trend-stats "med,p(95),p(99),max" load.js

# also save every single measurement, one JSON object per line, for your own analysis
k6 run --out json=results.json load.js

Checks and thresholds: pass or fail

Checks don't fail anything

It's easy to assume a failed check fails the test. It doesn't. Here's a script whose every single request hits a URL that doesn't exist (measured on this machine):

$ k6 run bad.js          # requests /api/nope, checks for status 200
    ✗ status is 200
      ↳  0% — ✓ 0 / ✗ 9417

    http_req_duration....: avg=150.69µs min=23.89µs med=122.99µs max=8.16ms p(90)=205.95µs p(95)=252.12µs
    http_req_failed......: 100.00% 9417 out of 9417

$ echo $?
0

100% of checks failed, 100% of requests failed — and k6 exited with 0, "success". Notice the latency too: a median of 0.12 ms, because a 404 Not Found takes no work. This is the "errors are fast" trap from §02 in real life. A test without thresholds can't fail.

Thresholds: your targets as rules

A ThresholdThreshold (k6)A pass/fail rule on a metric for the whole test, like p(95)<200 on http_req_duration. If any threshold is crossed, k6 marks the run failed and exits with code 99. is written in options.thresholds: a metric name, then a list of conditions that must hold over the whole test. Here are the targets from §02, written down:

import http from "k6/http";
import { check } from "k6";

export const options = {
  vus: Number(__ENV.VUS || 8), // pass a number in (k6 run -e VUS=64 load.js), or get 8
  duration: "30s",
  thresholds: {
    http_req_failed: ["rate<0.01"],                // fewer than 1% of requests fail
    http_req_duration: ["p(95)<100", "p(99)<250"], // the latency budget
    checks: ["rate>0.99"],                         // over 99% of checks pass
  },
};

export default function () {
  const res = http.get("http://localhost:8080/api/products");
  check(res, { "status is 200": (r) => r.status === 200 });
}

Two real runs against the §04 server, no sleep (measured on this machine). With 8 virtual users:

$ k6 run -e VUS=8 load.js
  █ THRESHOLDS

    checks
    ✓ 'rate>0.99' rate=100.00%

    http_req_duration
    ✓ 'p(95)<100' p(95)=25.34ms
    ✓ 'p(99)<250' p(99)=26.05ms

    http_req_failed
    ✓ 'rate<0.01' rate=0.00%
$ echo $?
0

With 64 virtual users, the same script — remember from §05 that 56 of them are standing in line at any moment:

$ k6 run -e VUS=64 load.js
level=error msg="thresholds on metrics 'http_req_duration' have been crossed"

  █ THRESHOLDS

    http_req_duration
    ✗ 'p(95)<100' p(95)=180.9ms
    ✓ 'p(99)<250' p(99)=193.47ms
    …
$ echo $?
99

One rule broken, so the whole run fails with code 99. (A program's exit code is the number it hands back when it ends; 0 means success, anything else means failure.) CI systems fail the job on any non-zero exit code, so this is all you need to make a pipeline go red.

Stop early when it's clearly failing

For long tests, you can stop the moment a threshold breaks instead of waiting an hour. Write the condition as an object:

thresholds: {
  http_req_duration: [
    { threshold: "p(95)<100", abortOnFail: true, delayAbortEval: "10s" },
  ],
},

abortOnFail: true stops the test as soon as the rule is broken; delayAbortEval: "10s" waits 10 seconds first so a slow first few requests during Warm-upWarm-upThe first minutes of a test, when caches are still empty, connections are being opened and (in some runtimes) code is still being optimized. Results from this period are usually slower than steady state. don't end the run. §09 uses this to find a server's limit automatically.

A smoke load test on every pull request

Big load tests are too slow for every push, but a small one isn't: 1–5 virtual users for 30 seconds against a freshly started server, with thresholds. It catches "this change made every request 10× slower" before it merges. The script is the same pattern as above, with a small fixed load and generous limits:

import http from "k6/http";
import { check, sleep } from "k6";

export const options = {
  vus: 2,
  duration: "30s",
  thresholds: {
    http_req_failed: ["rate<0.01"],
    http_req_duration: ["p(95)<200"], // generous: CI machines are noisy
    checks: ["rate>0.99"],
  },
};

export default function () {
  const res = http.get("http://localhost:8080/api/products");
  check(res, { "status is 200": (r) => r.status === 200 });
  sleep(0.5);
}

And a minimal CICI (continuous integration)A server that builds your code and runs its tests automatically on every push (GitHub Actions, GitLab CI, …), so mistakes are caught minutes after they're made. A flaky test there blocks everyone's merges. job in GitHub Actions that starts the server and runs it:

name: load-smoke
on: [pull_request]

jobs:
  k6:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-go@v7
        with:
          go-version: "1.26"
      - name: Start the server in the background
        run: go run . &
      - name: Wait until it answers
        run: timeout 60 sh -c 'until curl -sf localhost:8080/api/products; do sleep 1; done'
      - name: Smoke load test (fails the job if a threshold is crossed)
        run: docker run --rm -i --network host grafana/k6 run - < smoke.js

Open vs closed: how traffic arrives

The Closed workload modelClosed workload modelA fixed number of users; each one sends its next request only after the previous reply arrived. If the server slows down, the users automatically send less. is what §04–§06 used: vus: 10 means 10 loops of "send, wait for reply, repeat". The Open workload modelOpen workload modelNew requests arrive on a schedule (e.g. 300 per second) whether or not earlier ones have finished — like real visitors from the internet, who don't know your server is slow. instead fixes the Arrival rateArrival rateHow many new requests (or user journeys) start per unit of time, e.g. 300 per second. In an open-model test you choose it directly.: "start 300 new requests every second".

Real traffic to a public website or API is mostly open. Thousands of people you've never met each decide on their own when to click; none of them knows your server is slow right now. Closed traffic exists too — say, 4 batch workers that each process one job at a time — but it's the exception.

See the difference

Same simulated server as §01 (8 slots, ~20 ms each, so a ceiling of ~400/s). Try the closed model with 8, then 16, then 64 users. Then switch to open and try 300/s, then 450/s.

• Closed, many users: throughput is stuck at ~400/s and latency grows in proportion to the number of users (64 users ÷ 400/s ≈ 160 ms), but it never runs away. Only 64 requests can exist at once, so the line can never be longer than 64.
• Open, above capacity: the line grows by ~50 requests every second at 450/s, and latency keeps climbing for as long as the test runs. That's what would happen to real users — and a closed test can't show it.

Now compare how the two models see a freezing server (2 s frozen every 10 s — think of a long garbage-collection pause, or a database lock). Use the two preset buttons, which pick settings below the server's ceiling so the freeze is the only problem: closed with 8 users, then open at 300/s. Look at the "slow" count: in the simulation, about 0.2% of the closed test's requests waited over 500 ms, against nearly half of the open test's. Same server, same freezes. Hold that thought for §08.

Open-model tests in k6: scenarios and executors

To control the arrival rate, use a ScenarioScenario (k6)A named block in k6's options that says how traffic is generated: which executor, how many users or what arrival rate, and for how long. One test can run several scenarios at once. with an arrival-rate ExecutorExecutor (k6)The engine a k6 scenario uses to schedule work: e.g. constant-vus (a fixed number of looping users) or constant-arrival-rate (a fixed number of new iterations per second).:

import http from "k6/http";

export const options = {
  scenarios: {
    browse: {                              // any name you like
      executor: "constant-arrival-rate",   // open model: fixed arrivals per time unit
      rate: 300,                           // start 300 iterations…
      timeUnit: "1s",                      // …per second
      duration: "1m",
      preAllocatedVUs: 50,                 // VUs to create up front
      maxVUs: 500,                         // k6 may create more, up to this
    },
  },
};

export default function () {
  http.get("http://localhost:8080/api/products"); // no sleep: the rate sets the pace
}

The shapes from §03 work in the open model too: the ramping-arrival-rate executor takes the same kind of stages, but each target is a rate (requests per timeUnit) instead of a number of users. §09 uses it.

How many VUs will you need? Little's Law again: VUs busy ≈ arrival rate × latency. At 300/s and 21 ms, that's 300 × 0.021 ≈ 6 busy VUs. But if the server slows to 1 second per request, you'd need 300 × 1 = 300. Set maxVUs high enough for the bad case — that's the case you're testing for.

Real runs: open model against the §04 server

20 seconds at each rate, measured on this machine:

Compare with the closed table in §05: there, 64 users gave a calm-looking p95 of 173 ms. Here, asking for just 10% more than the server can do gives p95 near 1 second within 20 seconds — and it would keep growing in a longer run.

When k6 can't keep up: dropped iterations

The last column matters. When every VU is busy waiting for slow replies and k6 can't create new ones fast enough (or hits maxVUs), the iterations that were due are skipped and counted as Dropped iterationsDropped iterationsIn k6 arrival-rate tests: iterations that were due to start but couldn't, because no virtual user was free. A non-zero count means the test sent less load than you asked for.. In a run where maxVUs was 500, k6 also warned:

level=warning msg="Insufficient VUs, reached 500 active VUs and cannot initialize more" executor=constant-arrival-rate scenario=browse

Either message means the test sent less load than you asked for. That's still a useful finding (the server is past its limit), but don't report "it handled 450/s" — it didn't. Check dropped_iterations and vus_max in every arrival-rate summary.

Coordinated omission: when the tool lies

The name (popularized by Gil Tene, author of the HdrHistogramHdrHistogramA widely used library (Java, Go, JS and more) that records millions of latencies compactly with good precision, and can fill in samples lost to coordinated omission if you tell it the expected interval. library) describes what happens: the load tool and the server coordinate — without meaning to — to omit the worst measurements from the results.

The same server, measured two ways

The §04 server, changed so that every 10 seconds it freezes for 2 seconds (all requests wait — like a long PausePause (GC or VM)A moment when a program's runtime freezes all its threads, for example for garbage collection. From outside, a paused process looks exactly like a dead or unreachable one. or a table lock). The change is four lines at the top of the handler:

var t0 = time.Now() // when the server started

func products(w http.ResponseWriter, r *http.Request) {
	// NEW: from second 8 to second 10 of every 10 seconds, every request waits.
	if e := time.Since(t0) % (10 * time.Second); e >= 8*time.Second {
		time.Sleep(10*time.Second - e) // sleep until the freeze is over
	}
	// … the rest of products() exactly as in §04
}

time.Since(t0) % (10 * time.Second) is "how far we are into the current 10-second cycle" (% is the remainder after dividing). Between 8 s and 10 s into a cycle, the handler sleeps until the cycle ends.

Two k6 runs of 30 seconds each against it, measured on this machine:

Same server, same freezes, similar request counts. One test says p99 is 27 ms; the other says 2 seconds. The server was frozen 20% of the time, so about 20% of the open test's requests arrived during a freeze. Someone arriving at the start of a freeze waits almost 2 s, someone arriving at the end hardly at all, so that slowest 20% is spread from ~0 to 2 s. That's why the open test shows p90 ≈ 1 s (the middle of that spread) and p99 ≈ 2 s.

Why? The closed test has one virtual user. When the server freezes, that user's request is stuck for 2 s, and the user sends nothing else until it returns. So each freeze produces exactly one slow sample, out of ~380 per freeze cycle. The max column caught it; every percentile missed it.

On a timeline

Here's the effect up close: a client that plans to send a request every 50 ms. Compare measuring each request from when it was actually sent with measuring from when it should have been sent:

The fix: measure from the planned time

There are two ways out, and they're the same idea:

1. Use an open model (§07). Requests are sent on schedule, so the ones due during the freeze are sent — and they wait, and that wait is measured.
2. If you must loop, time each request from when it was supposed to start, not from when you got around to sending it.

Here's the second idea as a tiny load generator — a single loop that plans 20 requests per second and records both numbers. Both versions are complete programs:

package main

import (
	"fmt"
	"io"
	"net/http"
	"slices"
	"time"
)

func main() {
	const url = "http://localhost:8080/api/products"
	const rate = 20                // planned: 20 requests per second…
	const total = 600              // …for 30 seconds
	interval := time.Second / rate // = 50ms between planned sends

	var naive, corrected []time.Duration
	start := time.Now()
	for i := range total {
		planned := start.Add(time.Duration(i) * interval)
		time.Sleep(time.Until(planned)) // negative = already late: returns at once

		sent := time.Now()
		get(url)
		done := time.Now()

		naive = append(naive, done.Sub(sent))            // from when we actually sent it
		corrected = append(corrected, done.Sub(planned)) // from when it SHOULD have been sent ← the point
	}
	fmt.Println("naive:    ", summary(naive))
	fmt.Println("corrected:", summary(corrected))
}

// get sends one GET request and reads the whole reply.
func get(url string) {
	resp, err := http.Get(url)
	if err != nil {
		return
	}
	io.Copy(io.Discard, resp.Body) // read the body so the request really finished
	resp.Body.Close()
}

// pct returns the p-th percentile (0–100) of ds: sort a copy, pick the value at that rank.
func pct(ds []time.Duration, p int) time.Duration {
	s := slices.Clone(ds)
	slices.Sort(s)
	i := (len(s)*p+99)/100 - 1 // rank p% of the way up, rounded up
	return s[max(i, 0)].Round(time.Millisecond)
}

func summary(ds []time.Duration) string {
	return fmt.Sprintf("p50=%v p99=%v", pct(ds, 50), pct(ds, 99))
}

Against the freezing server, the Go version printed this (measured on this machine; the JS version gave p99 values of 28 ms and 1926 ms):

naive:     p50=21ms p99=29ms
corrected: p50=24ms p99=1.923s

Same loop, same requests, same server. Only the starting point of the stopwatch changed.

Staff-level notes

• Open-model tools can omit too. If k6 runs out of VUs, it drops the iterations that were due (§07). Those are exactly the requests that would have waited the longest. A run with many dropped_iterations has the same blind spot — treat its percentiles as optimistic.
• Some tools correct for it. HdrHistogram can fill in the missing samples if you tell it the expected interval, and wrk2 is a version of the classic wrk benchmark tool rewritten to send at a constant rate. Check what your tool does before trusting its p99.
• Your monitoring can have the same blind spot. Server-side latency metrics often start the timer when the handler starts — after the request already waited in a connection queue. Measure as close to the user as you can.
• "Max" is a hint. When the max is 100× the p99, something stalled and most of the stall may be missing from the percentiles.

Finding the knee

Think of a supermarket with 8 checkouts. At quiet times you walk straight to an open one. As it fills up, now and then you wait behind someone. When all 8 are busy nearly all the time, every new shopper waits behind several others — and if shoppers arrive faster than the checkouts can serve them, the line grows without end.

Real numbers from the §04 server

The KneeKnee (of the latency curve)The load level where latency stops being flat and starts climbing steeply, because the busiest resource is nearly always full. Past it, extra traffic only makes the waiting line longer. is easiest to see when you test several fixed arrival rates, one after another, and plot two things for each: how much got done, and how long it took.

Three zones show up in every system:

1. Below the knee (up to ~350/s here): throughput equals what you asked for; latency equals "no load" latency. Adding traffic costs nothing.
2. At the knee (~380/s): still keeping up, but the tail starts to rise (p95 25 → 30 ms). Requests sometimes find all 8 slots busy and wait a moment.
3. Past the knee (400/s and up): throughput is flat at the ceiling (~388/s), and latency jumps from milliseconds to seconds. More traffic only makes the line longer.

Why the bend is so sharp

Requests don't arrive in a perfectly even stream; they come in random little clumps. At 50% UtilizationUtilizationThe fraction of time workers are busy. 80% utilization means that, on average, 8 of 10 workers are working at any moment., a clump usually finds free slots. At 95%, almost every request arrives to find all slots busy and has to wait behind the ones already waiting. The waiting time doesn't grow in a straight line with busyness — it grows faster and faster as you approach 100%. Worker pools: why latency explodes near 100% busy shows the curve and the formula.

A breakpoint test finds it automatically

Instead of separate runs, you can ramp the arrival rate steadily and let a threshold stop the test (§06):

import http from "k6/http";

export const options = {
  scenarios: {
    breakpoint: {
      executor: "ramping-arrival-rate",
      startRate: 50,                               // begin at 50 requests per second…
      timeUnit: "1s",
      preAllocatedVUs: 50,
      maxVUs: 1000,
      stages: [{ duration: "2m", target: 600 }],   // …and climb steadily to 600/s
    },
  },
  thresholds: {
    http_req_duration: [
      { threshold: "p(95)<100", abortOnFail: true, delayAbortEval: "10s" },
    ],
  },
};

export default function () {
  http.get("http://localhost:8080/api/products");
}

stages here lists targets for the rate, not for users: over 2 minutes, go from startRate up to 600 per second. Against the §04 server (measured on this machine):

running (1m20.0s), 0000/0077 VUs, 18557 complete and 77 interrupted iterations
breakpoint ✗ [  67% ] 0075/0077 VUs  1m20.0s/2m0s  416.61 iters/s
level=error msg="thresholds on metrics 'http_req_duration' were crossed; at least one has abortOnFail enabled, stopping test prematurely"

    ✗ 'p(95)<100' p(95)=122.74ms

It stopped at 1 minute 20 seconds, when the rate was 50 + 550 × (80 ÷ 120) ≈ 417/s (k6 itself shows 416.61 iterations/s) — noticeably past the real knee of ~380/s. Why so late? A threshold is computed over the whole test so far. The first minute of fast requests keeps the overall p95 low, so the slow requests at the end need a while to push it over 100 ms.

Then ask: what was full?

A knee is always caused by one resource running out — the BottleneckBottleneckThe one resource that runs out first under load — CPU, a connection pool, a lock, a slow dependency. It sets the maximum throughput for the whole system.. Here we know it: the pool of 8 slots, 8 ÷ ~20 ms ≈ 400/s. In a real system, find it by watching server-side metrics during the test: which one hits 100% or starts to build a queue right at the knee? CPU? The database Connection poolConnection poolA set of open database (or HTTP) connections that your program keeps and reuses instead of opening a new one per request. Pools usually have a size limit; when every connection is busy, the next caller waits for one to be returned.? A lock? A downstream service?

Fixing it moves the knee. With the pool raised to 16 slots, the same server measured on this machine kept up at 760/s (p95 61 ms) and hit its ceiling at about 776/s — roughly double, as the arithmetic predicts. A higher rate would eventually expose the next bottleneck (CPU, the network, the database itself). Here even k6 started to drop a few iterations at 700/s, because the load generator and the server were sharing the same 4-CPU machine — §11 explains why that's a problem.

Capacity planning: from knee to server count

Back to the bridge: it bends dangerously at 60 tonnes, so the sign says 40. Nobody posts the breaking point as the limit. Capacity planningCapacity planningWorking out how many servers (or how much database, cache, bandwidth) you need for the traffic you expect, plus a safety margin — using numbers measured in load tests. works the same way: measure where it bends, then plan to stay well below that line.

A worked example

• Safe rate per instance. §09 measured a knee of about 380/s for one instance while p95 stayed under 100 ms. We'll only plan to run each instance at 70% of that: 380 × 0.7 = 266/s. The other 30% is HeadroomHeadroomSpare capacity you keep on purpose — e.g. workers that are only 80% busy on average — so random bursts can be cleared quickly instead of piling up. for random bursts.
• Peak traffic. Your metrics say the busiest minute last month was 2,400 requests/s. (Not the daily average! Traffic at 8 p.m. can easily be several times the 4 a.m. level.)
• Growth. Marketing expects 30% more users before the next review: 2,400 × 1.3 = 3,120/s.
• Instances: 3,120 ÷ 266 = 11.7 → round up to 12.
• Spare: add one so a crash or a Rolling deployRolling deployReplacing the instances of an app a few at a time: start a new one, wait until it's ready, drain and stop an old one, repeat. The service stays up throughout. (servers replaced a few at a time) doesn't push the rest over the edge: 13 instances. This is called N+1.

The traps in each input

"Per instance" assumes adding instances adds capacity. Often it doesn't, fully. Ten app servers might share one database, one cache, one rate-limited payment API. Scale the app tier and the bottleneck simply moves there. Before you trust the multiplication, run the test against 2 and then 4 instances behind the Load balancerLoad balancerA component that spreads incoming requests or connections across several servers, so no single one is overloaded and a dead one can be skipped.: if 4 instances don't give close to 4× the throughput, find out what's shared. See Load balancing.

Count the shared resources too. Little's LawLittle's LawL = λ × W: the average number of things in a system equals the rate they arrive times how long each one stays. 100 requests/s × 0.2 s each = 20 in progress at any moment. tells you how many requests are in flight at once: 3,120 req/s × 0.02 s ≈ 62 requests using a database connection at any moment, on average. If the database allows 100 connections and every instance opens a pool of 20, then 13 instances × 20 = 260 connections — far more than the database accepts. Load tests at the planned scale catch this; spreadsheets don't.

Peaks have peaks. A per-minute peak hides per-second bursts. That's what the headroom is for — and why running at 95% of the knee isn't "efficient", it's fragile.

Auto-scaling isn't instant. AutoscalingAutoscalingAutomatically adding instances when a metric (CPU, requests per second, latency) is high and removing them when it's low. adds servers when load rises, but noticing the rise, starting a machine and warming it up can take minutes. Your headroom must carry the traffic during those minutes. A spike test (§03) tells you whether it does.

Testing somewhere that resembles production

Testing a bridge model in a wind tunnel is useful — but only if the model has the same shape, materials and proportions as the real bridge. Here's what has to match, roughly in order of how often it goes wrong.

1. The environment

Run against a staging environmentStaging environmentA copy of production — same code, similar machines and data — used to test changes before they reach real users. Load tests usually run here. that matches production in shape: the same instance sizes, the same number of database replicas, the same load balancer, the same configuration (connection-pool sizes, timeouts, log levels). A staging database on a tiny machine will become the bottleneck at a tenth of production's traffic and send you chasing the wrong problem.

If staging must be smaller, keep the ratios the same (one app server per database replica, for example) and test one instance's capacity, which you then multiply carefully (§10).

2. The data

A query that takes 2 ms on a table of 1,000 rows can take 2 seconds on 50 million — especially if an index is missing (see Indexes). Load-test against production-sized data: an anonymized copy of production (real data with names, emails and other personal details replaced by fakes), or generated data with realistic sizes and distributions.

Also vary what each virtual user asks for. If every VU requests product #1, it's served from the CacheCacheA small, fast store of answers you already computed or fetched, so the next request for the same thing skips the slow work. The hard part is knowing when a cached answer is out of date. every time and you're measuring the cache. Real users ask for thousands of different things, and many of those miss the cache. Pick IDs at random from a realistic list.

3. The dependencies

Your service calls a payment provider, an email service, a search cluster. Options:

• Call the real staging versions when they can take the load (and their owners agree).
• Replace them with a fake that answers with realistic delays, not instantly — a fake that responds in 0 ms makes your service look faster than it will ever be. See Test doubles.
• Never send load to a third party's production API.

4. The load generator

The Load generatorLoad generatorThe machine (or program) sending the test traffic. If it runs out of CPU or network itself, your results measure the generator, not the system under test. is a computer too. If it runs out of CPU or network, requests leave late and timings get inflated. Watch it during the test:

• Don't run it on the same machine as the server. They fight over the same CPUs. (§09's 700/s run already showed a few dropped iterations for exactly this reason.)
• Check its CPU stays well below 100%. If not, use a bigger machine or several generators. k6 can split one test across machines; Grafana Cloud k6 (a hosted service) and the k6 operator for KubernetesKubernetesA system that runs your containers across many machines: it decides where each one runs, restarts them when they crash, rolls out new versions, and gives each group a stable name other services can call. (an add-on that runs one test as many k6 copies on a cluster) do this for you.
• Mind the network distance. A generator in the same data centre adds ~1 ms; one across an ocean adds ~100 ms to every request. Neither is "wrong", but know which question you're asking: "how fast is the server?" or "how fast is it for users in Asia?"

5. Time

Run long enough to get past Warm-upWarm-upThe first minutes of a test, when caches are still empty, connections are being opened and (in some runtimes) code is still being optimized. Results from this period are usually slower than steady state. and to see slow effects: garbage collectionGarbage collector (GC)Part of the language runtime that automatically frees memory nobody can reach anymore. Anything still referenced — like a stuck goroutine — is never freed. cycles, cache expiry, connection recycling, log rotation. A 30-second test is fine for a smoke test; a capacity measurement wants several minutes at each step; leaks need a soak test of hours (§03).

Testing in production

Sometimes staging can never be realistic enough. Teams then test in production, carefully:

• Off-peak, with a kill switch. Run during quiet hours, with someone watching dashboards and a way to stop the test instantly.
• Marked traffic. Add a header such as X-Load-Test: 1 so test requests are easy to filter out of analytics, skip real payments and emails, and can be blocked at the edge (the load balancer or CDN that receives traffic before your servers do) if something goes wrong.
• Shadow trafficShadow traffic (dark launch)Sending a copy of real requests to a new system and comparing its answers with the old one's, while users still get the old system's answers.. Copy a share of real requests to a new version and throw its answers away. Real traffic shapes, no risk to users.
• Start small. Production tests usually aim at "can we handle 1.5× today's peak?", not "where does it break?" Breakpoint tests belong in staging.

Pitfalls gallery

export default function () {
  // every request: same product, always cached
  http.get(`${BASE}/api/products?id=1`);
}

export default function () {
  const id = Math.floor(Math.random() * 10_000) + 1; // a random product, 1–10,000
  http.get(`${BASE}/api/products?id=${id}`, {
    tags: { name: "GET /api/products?id=:id" }, // one name for all of them in the results
    timeout: "5s",                              // give up like a real client would (see #8)
  });
}

Check yourself

Predict the answer before you click. Each explanation points back to the section that covers it.

Cheat sheet