End-to-End Testing

The 30-second version

What actually runs

Our example the whole way through is the Tiny Shop: a sign-in form, one product (a rubber duck, $5), a cart and a Pay button. The page is plain HTML and JavaScript; the server is a small Go program. A test tool called PlaywrightPlaywrightA library that drives real browsers (Chromium, Firefox, WebKit) from code, used for end-to-end tests and automation. drives a real Chromium browser, usually a headlessHeadless ChromiumChrome's engine run by a script with no visible window. Test tools like Playwright use it to click, type and measure exactly as a real browser would. one with no window on screen.

Pick a step of a checkout test and see which parts of the system it touches:

The note under click “Pay” is the whole trade-off in one sentence. An E2E test is the most realistic test you can write: nothing is faked, so it catches the bugs that live between parts (the button posts to the wrong URL, the server returns cents but the page expects dollars). It's also the slowest and the vaguest: seconds per test instead of milliseconds, and a red result could be any layer.

What this topic covers

1. What deserves an E2E test (§02): a handful of critical journeysCritical user journeyA path through the app that must never break because the business depends on it: sign up, log in, pay, the core thing users come for. These are the paths worth an end-to-end test., not everything.
2. Writing one with Playwright (§03–§05): your first test, finding elements the way users do, and the auto-waiting that keeps tests from being flakyFlaky testA test that sometimes passes and sometimes fails without any code change — usually because it depends on the real clock, randomness, timing or the network..
3. Keeping a suite tidy (§06–§08): page objects, fixtures, test data and logging in once.
4. Running it for real (§09–§11): reading traces when a test fails, CI against a production build, and screenshot comparison in brief.
5. Staff level (§12–§13): an honest look at this site's own browser smoke testSmoke testA quick, broad check that the app starts and its main pages load and respond without crashing. It doesn't check that results are correct, only that nothing is obviously on fire., and how to keep a suite healthy as it grows.

What deserves an E2E test

Why not test everything end to end?

It's tempting: E2E tests are the most realistic, so why not write hundreds? Two numbers explain why teams that try it regret it.

Time. Say each E2E test takes about 10 seconds (open a page, sign in, click around, wait for the server). 200 of them, one after another, take 33 minutes. Running 4 at once brings that to about 8 minutes, on every push, for every developer. The same 200 checks as unit tests would take well under a second.

Flakiness. A Flaky testFlaky testA test that sometimes passes and sometimes fails without any code change — usually because it depends on the real clock, randomness, timing or the network. is one that sometimes fails with no code change. Browsers, networks and timing make E2E tests the flakiest kind. Suppose each test fails randomly just 1 time in 200 (0.5%). Then the chance that all 200 pass in one run is 0.995²⁰⁰ ≈ 37%. Nearly two runs out of three go red for no reason, and people stop believing the red.

What earns an E2E test

Ask of each candidate: "If this broke in production, how bad would it be, and could a cheaper test catch it?" E2E tests are for the "very bad, and no" corner:

• Critical journeys. Sign up, sign in, the purchase or the core action users come for, password reset. If one of these breaks, you lose money or users within minutes. These are the critical user journeysCritical user journeyA path through the app that must never break because the business depends on it: sign up, log in, pay, the core thing users come for. These are the paths worth an end-to-end test..
• Wiring only a browser can see. The form posts the right fields, the page reads the JSON the server actually sends, the cookie from sign-in is really sent on the next request.
• Browser differences. "Works in Chrome" isn't "works in Safari". Run the critical journeys in more than one browser engine.
• Bugs that already escaped once across layers. A Regression testRegression testA test added after a bug is fixed, so the same bug can't quietly come back. at the E2E level is fine when no lower-level test can reproduce the bug.

What doesn't

• Variations of logic. 30 discount cases, 15 bad email formats, tax for 40 countries: test those as fast unit tests. One E2E test can prove the discount box is wired up; it shouldn't prove the arithmetic.
• Every error message. Test the validation function directly; at most, one E2E step checks that an error shows up.
• Systems you don't own. Don't let your suite depend on a real payment provider being up. Use the provider's test mode or a stand-in at the edge (see Mocks, Stubs & Fakes §08).
• Things nobody would miss. The footer year. Why & What to Test §07 has more on what not to test at all.

Try it on the Tiny Shop:

One journey, several checks

A good E2E test usually reads like a short story: sign in, add a duck, pay, see the confirmation. It's fine for it to check a few things along the way, such as the cart count after adding. What you avoid is the opposite: twenty tiny E2E tests that each sign in from scratch to check one label. Each one pays the full cost of starting a browser and signing in.

Your first Playwright test

The app we're testing

The Tiny Shop's page is small. These are the parts the test will touch. Notice the visible labels and the role and data-testid attributes; §04 explains why they matter:

<section id="login">
<label>Email <input name="email" type="email"></label>
<label>Password <input name="password" type="password"></label>
<button type="submit">Sign in</button>
</section>

<h2>Rubber duck</h2>
<button id="add">Add to cart</button>
<p>Cart: <span data-testid="cart-count">0</span> items</p>
<button id="pay">Pay</button>
<p role="status" id="status"></p>   <!-- "Paying…", then "Order … confirmed: $5" -->

The server is a Go program with a few JSON endpointsEndpointOne operation an API exposes, usually a URL plus a method — e.g. GET /orders/42 or POST /payments.: POST /api/login, POST /api/cart and POST /api/orders (which pretends the payment provider takes 700 ms). The test drives the page; the page calls the server; nothing is faked.

// Tiny Shop: a small Go server to test end to end. Sign in, add to cart, pay.
package main

import (
	"crypto/rand"
	"encoding/hex"
	"encoding/json"
	"log"
	"net/http"
	"os"
	"sync"
	"time"
)

type store struct {
	mu       sync.Mutex
	users    map[string]string // email → password
	sessions map[string]string // session id → email
	carts    map[string]int    // email → items in cart
}

var db = &store{users: map[string]string{}, sessions: map[string]string{}, carts: map[string]int{}}

func newID() string {
	b := make([]byte, 16)
	rand.Read(b)
	return hex.EncodeToString(b)
}

// currentUser looks up the session cookie and returns the signed-in email.
func currentUser(r *http.Request) (string, bool) {
	c, err := r.Cookie("sid")
	if err != nil {
		return "", false
	}
	db.mu.Lock()
	defer db.mu.Unlock()
	email, ok := db.sessions[c.Value]
	return email, ok
}

func main() {
	mux := http.NewServeMux()
	mux.Handle("GET /", http.FileServer(http.Dir("public"))) // serves public/index.html

	mux.HandleFunc("POST /api/login", func(w http.ResponseWriter, r *http.Request) {
		var in struct{ Email, Password string }
		json.NewDecoder(r.Body).Decode(&in)
		db.mu.Lock()
		ok := in.Password != "" && db.users[in.Email] == in.Password
		sid := newID()
		if ok {
			db.sessions[sid] = in.Email
		}
		db.mu.Unlock()
		if !ok {
			http.Error(w, "wrong email or password", http.StatusUnauthorized)
			return
		}
		http.SetCookie(w, &http.Cookie{Name: "sid", Value: sid, Path: "/", HttpOnly: true})
		w.WriteHeader(http.StatusNoContent)
	})

	mux.HandleFunc("GET /api/me", func(w http.ResponseWriter, r *http.Request) {
		if _, ok := currentUser(r); !ok {
			http.Error(w, "not logged in", http.StatusUnauthorized)
			return
		}
		w.WriteHeader(http.StatusNoContent)
	})

	mux.HandleFunc("POST /api/cart", func(w http.ResponseWriter, r *http.Request) {
		email, ok := currentUser(r)
		if !ok {
			http.Error(w, "log in first", http.StatusUnauthorized)
			return
		}
		db.mu.Lock()
		db.carts[email]++
		n := db.carts[email]
		db.mu.Unlock()
		json.NewEncoder(w).Encode(map[string]int{"items": n})
	})

	mux.HandleFunc("POST /api/orders", func(w http.ResponseWriter, r *http.Request) {
		email, ok := currentUser(r)
		if !ok {
			http.Error(w, "log in first", http.StatusUnauthorized)
			return
		}
		db.mu.Lock()
		n := db.carts[email] // the total is decided now…
		db.carts[email] = 0
		db.mu.Unlock()
		time.Sleep(700 * time.Millisecond) // …then the pretend payment provider is slow
		json.NewEncoder(w).Encode(map[string]any{"order": newID()[:8], "total": n * 5})
	})

	if os.Getenv("APP_ENV") == "test" { // ← test-only routes never exist in production
		mux.HandleFunc("POST /test/users", func(w http.ResponseWriter, r *http.Request) {
			var in struct{ Email, Password string }
			if err := json.NewDecoder(r.Body).Decode(&in); err != nil || in.Email == "" || in.Password == "" {
				http.Error(w, "need email and password", http.StatusBadRequest)
				return
			}
			db.mu.Lock()
			db.users[in.Email] = in.Password
			db.mu.Unlock()
			w.WriteHeader(http.StatusCreated)
		})
	}

	addr := ":" + envOr("PORT", "8080")
	log.Println("listening on", addr)
	log.Fatal(http.ListenAndServe(addr, mux))
}

func envOr(k, def string) string {
	if v := os.Getenv(k); v != "" {
		return v
	}
	return def
}

<!doctype html>
<html lang="en">
<head><meta charset="utf-8"><title>Tiny Shop</title>
<style>body{font-family:sans-serif;max-width:480px;margin:2rem auto}</style></head>
<body>
<h1>Tiny Shop</h1>
<section id="login">
  <h2>Sign in</h2>
  <form id="login-form">
    <label>Email <input name="email" type="email"></label><br>
    <label>Password <input name="password" type="password"></label><br>
    <button type="submit">Sign in</button>
    <p role="alert" id="login-error"></p>
  </form>
</section>
<section id="shop" hidden>
  <h2>Rubber duck</h2>
  <p>$5</p>
  <button id="add">Add to cart</button>
  <p>Cart: <span data-testid="cart-count">0</span> items</p>
  <button id="pay">Pay</button>
  <p role="status" id="status"></p>
</section>
<script>
const $ = (s) => document.querySelector(s);
const showShop = () => { $("#login").hidden = true; $("#shop").hidden = false; };

$("#login-form").addEventListener("submit", async (e) => {
  e.preventDefault();                                   // don't reload the page
  const f = new FormData(e.target);
  const res = await fetch("/api/login", { method: "POST", body: JSON.stringify({ email: f.get("email"), password: f.get("password") }) });
  if (!res.ok) { $("#login-error").textContent = "Wrong email or password"; return; }
  showShop();
});
$("#add").addEventListener("click", async () => {
  const { items } = await (await fetch("/api/cart", { method: "POST" })).json();
  $("[data-testid=cart-count]").textContent = items;
});
$("#pay").addEventListener("click", async () => {
  $("#status").textContent = "Paying…";
  const o = await (await fetch("/api/orders", { method: "POST" })).json();
  $("#status").textContent = `Order ${o.order} confirmed: $${o.total}`;
});
fetch("/api/me").then((r) => { if (r.ok) showShop(); }); // already signed in? (cookie)
</script>
</body></html>

Install and configure

In a Node project, npm init playwright@latest asks a few questions, then creates a playwright.config.js, an example test, and downloads the browsers. (Or add it by hand: npm install -D @playwright/test, then npx playwright install chromium.) The config file tells Playwright where the tests are and how to start your app:

import { defineConfig, devices } from "@playwright/test";

export default defineConfig({
  testDir: "./tests",
  fullyParallel: true,                    // run tests in parallel, even within one file
  use: {
    baseURL: "http://localhost:8080",     // ← so tests can write page.goto("/")
  },
  projects: [
    { name: "chromium", use: { ...devices["Desktop Chrome"] } },
  ],
  webServer: {                            // ← Playwright starts the app for you
    command: "go run .",                  //   compile and start the Go server…
    cwd: "./app",                         //   …from the app's folder
    url: "http://localhost:8080/",        //   waits until this URL answers
    env: { APP_ENV: "test" },             //   turns on test-only endpoints (§08)
    reuseExistingServer: !process.env.CI, //   locally, reuse one you already started
    timeout: 120_000,                     //   the first go run also compiles: allow 2 min
  },
});

webServer is the piece beginners miss. Without it you must remember to start the server yourself before every run. With it, npx playwright test starts the app, waits until url answers, runs the tests, and stops the server afterwards. (If command fails, say the binary doesn't exist, the run stops with the command's error before any test starts.) §10 swaps go run . for a binary built once in CI. A project is one way of running the whole suite, here "in desktop Chrome"; §10 adds more.

The test

import { test, expect } from "@playwright/test";

test("a signed-in user can buy a duck", async ({ page, request }) => {
  const email = `ana+${Date.now()}@example.com`;                       // a unique user per run
  const res = await request.post("/test/users", { data: { email, password: "s3cret" } });
  await expect(res).toBeOK();                                          // ← stop here if seeding failed

  await page.goto("/");
  await page.getByLabel("Email").fill(email);
  await page.getByLabel("Password").fill("s3cret");
  await page.getByRole("button", { name: "Sign in" }).click();

  await page.getByRole("button", { name: "Add to cart" }).click();
  await expect(page.getByTestId("cart-count")).toHaveText("1");        // ← waits for the update

  await page.getByRole("button", { name: "Pay" }).click();
  await expect(page.getByRole("status")).toHaveText(/Order \w+ confirmed: \$5/);
});

Run it

$ npx playwright test
[WebServer] 2026/10/30 22:18:09 listening on :8080

Running 1 test using 1 worker

  ✓  1 [chromium] › tests/checkout.spec.js:3:5 › a signed-in user can buy a duck (1.1s)

  1 passed (2.0s)

That's real output from Playwright 1.63 on this machine (the [WebServer] line is the Go server's own log). Now step through what each line made happen:

1test("a signed-in user can buy a duck", async ({ page, request }) => {
2  const email = `ana+${Date.now()}@example.com`;
3  const res = await request.post("/test/users", { data: { email, password: "s3cret" } });
4  await expect(res).toBeOK();
5 
6  await page.goto("/");
7  await page.getByLabel("Email").fill(email);
8  await page.getByLabel("Password").fill("s3cret");
9  await page.getByRole("button", { name: "Sign in" }).click();
10 
11  await page.getByRole("button", { name: "Add to cart" }).click();
12  await expect(page.getByTestId("cart-count")).toHaveText("1");
13 
14  await page.getByRole("button", { name: "Pay" }).click();
15  await expect(page.getByRole("status")).toHaveText(/Order \w+ confirmed: \$5/);
16});

Watching it

Headless runs are fast but invisible. While writing a test you'll want to see it:

Locators: find things the way users do

Locators are descriptions, not elements

In Playwright you find things with a LocatorLocator (Playwright)A description of how to find an element on the page, such as getByRole('button', { name: 'Pay' }). It doesn't search right away: Playwright looks for the element fresh every time you use the locator, so it still works after the page re-renders.:

const pay = page.getByRole("button", { name: "Pay" }); // ← nothing is searched yet
await pay.click();                                      // searches now, then clicks
await expect(pay).toBeEnabled();                        // searches again, fresh

A locator is a recipe for finding an element, not the element itself. Playwright runs the recipe every time you use it, so it keeps working when React re-renders the button or the page swaps the whole section. Older tools handed you an element once and failed with "stale element" errors when the page replaced it.

The ways to describe an element

Roughly in the order Playwright's own docs recommend:

The role is what kind of thing an element is (button, link, heading); the accessible name is what a screen reader would announce for it, usually its visible text or label. The browser works both out from your HTML; Accessibility §04 shows how. That's why role locators are so good: they find elements the same way assistive technology does.

By default, name and text matching are case-insensitive and match a substring, so { name: "pay" } also matches "Pay now". Add exact: true when that's too loose.

See what breaks

Below is a real mini page and real queries run against it. Tick the changes a teammate might make next sprint:

The pattern to notice: CSS and XPath locators break on changes users can't see, and those are the failures that make people hate E2E tests. Role locators break only when something a user would notice changes. When they break on the <div> change, they've caught a real accessibility bug.

Strict mode: exactly one, please

Actions like click() and fill() insist the locator matches exactly one element. This is Strict modeStrict mode (locators)Playwright's rule that an action like click() or fill() must match exactly one element. If the locator matches two, the action fails with “strict mode violation” instead of guessing which one you meant.. Here's what Playwright printed when we asked for "a button" on the shop page, which has two:

Error: locator.click: Error: strict mode violation: getByRole('button') resolved to 2 elements:
    1) <button id="add">Add to cart</button> aka getByRole('button', { name: 'Add to cart' })
    2) <button id="pay">Pay</button> aka getByRole('button', { name: 'Pay' })

It refuses to guess, and even suggests better locators. When the page has a list of similar things (imagine the shop grew a product list, as in the lab above), scope the locator, the way you'd say "the button in the rubber duck card":

await page
  .getByRole("listitem")                                // every <li> in the list…
  .filter({ hasText: "Rubber duck" })                   // …keep the one mentioning the duck…
  .getByRole("button", { name: "Add to cart" })         // …and the button inside it
  .click();

When a test id is the right call

data-testid attributes (test idsTest id (data-testid)An HTML attribute added only so tests can find an element: <span data-testid="cart-count">. Users never see it, so text or style changes don't break it — but neither does an accessibility bug.) exist only for tests, so they survive any restyle or rewording. Use them when there's nothing better: the Tiny Shop's cart count is just a number in a <span>, with no label of its own, so it gets data-testid="cart-count". Other good cases: rows of data with no unique visible text, or an app whose wording is translated into many languages.

Don't make them the default. As the lab showed, a test id keeps "finding the button" even after it became an unusable <div>.

// tied to the page's structure and styling
await page.locator("#app > div:nth-child(2) .btn-primary").click();
await page.locator("//form/div[3]/input").fill("ana@example.com");

// tied to what the user sees
await page.getByRole("button", { name: "Add to cart" }).click();
await page.getByLabel("Email").fill("ana@example.com");

Auto-waiting and web-first assertions

The problem: the page isn't ready yet

Almost everything on a web page happens later: after a request comes back, after an animation, after React re-renders. In the Tiny Shop, clicking Pay shows "Paying…" and only ~700 ms later "Order … confirmed". Here's a test that reads the text right after clicking, and its real failure:

await page.getByRole("button", { name: "Pay" }).click();
expect(await page.getByRole("status").textContent()).toContain("confirmed"); // read once

Error: expect(received).toContain(expected) // indexOf

Expected substring: "confirmed"
Received string:    "Paying…"

The classic "fix" is to add await page.waitForTimeout(500) (a sleep) before reading. We tried that too: it failed the same way, because 500 ms is less than 700 ms. So people bump it to 1000, then 3000 on slow CI… and the suite gets slower and still flaky. Playwright's docs say plainly that waitForTimeout is for debugging only and that tests relying on it are inherently flaky.

Actions wait for you

Every action (click, fill, check, …) runs actionability checksActionability checksThe conditions Playwright waits for before acting on an element. For a click: it is visible, enabled, stable (not animating), and nothing else sits on top of it to catch the click. first and retries until they pass. This is Auto-waitingAuto-waitingBefore every action, Playwright waits until the element is ready (visible, enabled, not moving, not covered) instead of failing straight away. That removes most of the sleep() calls older browser tests needed.. Step through a deliberately awkward page:

1// what you write:
2await page.getByRole("button", { name: "Pay" }).click();
3 
4// what Playwright does for you, roughly:
5// 1. find exactly one matching element
6// 2. wait until it is visible
7// 3. wait until it is enabled
8// 4. wait until it is stable (not moving)
9// 5. scroll it into view
10// 6. check nothing covers it, then click
11//    (if any check fails: wait a moment, retry)

When a check never passes, you get a call log telling you which one. This is a real one. The test called getByRole("button") right after clicking Sign in, at the instant the sign-in form was the only thing on screen, so it matched just the Sign in button. A moment later the form was hidden, and Playwright waited for a button that would never be visible again:

Error: locator.click: Test timeout of 30000ms exceeded.
Call log:
  - waiting for getByRole('button')
    - locator resolved to <button type="submit">Sign in</button>
  - attempting click action
    2 × waiting for element to be visible, enabled and stable
      - element is not visible
    - retrying click action

Assertions that wait: web-first

The second half is the assertion. Compare:

const text = await page.getByRole("status").textContent();
expect(text).toContain("confirmed");

expect(await page.getByText("Saved").isVisible()).toBe(true);

await expect(page.getByRole("status")).toContainText("confirmed");


await expect(page.getByText("Saved")).toBeVisible();

On the left, you pull a value out of the page and hand it to a plain expect, which has no idea where it came from. It checks once. On the right, you hand expect the locator itself, so it can go back to the page and look again, and again, for up to 5 seconds by default. That's a Web-first assertionWeb-first assertionAn assertion like await expect(locator).toHaveText('Saved') that keeps re-checking the live page until it passes or a timeout (5 s by default) runs out. The opposite of reading a value once and checking it.. Note the await in front of expect on the right: without it, the test doesn't wait for the re-checking to finish.

When one fails, the call log shows what it saw each time. Real output, checking for "2" when the cart had 1:

Error: expect(locator).toHaveText(expected) failed

Locator:  getByTestId('cart-count')
Expected: "2"
Received: "1"
Timeout:  5000ms

Call log:
  - Expect "toHaveText" getByTestId('cart-count') with timeout 5000ms
  - waiting for getByTestId('cart-count')
    - locator resolved to <span data-testid="cart-count">0</span>
    - unexpected value "0"
    13 × locator resolved to <span data-testid="cart-count">1</span>
       - unexpected value "1"

It saw "0", then "1" thirteen times, then gave up after 5 seconds. The ones you'll use most: toBeVisible, toHaveText, toContainText, toHaveValue, toBeEnabled, toHaveCount, toHaveURL.

Try it: sleeps vs waiting for the thing

Timeouts, and what not to do with them

There are two clocks to know. The test timeout (30 s by default) limits a whole test, including every auto-waiting action. The expect timeout (5 s by default) limits each web-first assertion. You can raise them in the config (timeout: 60_000, expect: { timeout: 10_000 }) or for one slow step (toHaveText("…", { timeout: 15_000 })).

Page objects: one place that knows the page

The problem it solves

After a few weeks you'll have this sign-in block copied into every test:

await page.goto("/");
await page.getByLabel("Email").fill(email);
await page.getByLabel("Password").fill("s3cret");
await page.getByRole("button", { name: "Sign in" }).click();

Then the designer renames "Sign in" to "Log in" and twenty tests go red. Every one needs the same one-word fix.

A page object for the shop

export class ShopPage {
  constructor(page) {
    this.page = page;
    this.addButton = page.getByRole("button", { name: "Add to cart" }); // locators live here, once
    this.payButton = page.getByRole("button", { name: "Pay" });
    this.cartCount = page.getByTestId("cart-count");
    this.status = page.getByRole("status");
  }

  async signIn(email, password) {
    await this.page.goto("/");
    await this.page.getByLabel("Email").fill(email);
    await this.page.getByLabel("Password").fill(password);
    await this.page.getByRole("button", { name: "Sign in" }).click();
  }

  async addToCart(times = 1) {
    for (let i = 0; i < times; i++) {
      const saved = this.page.waitForResponse("**/api/cart"); // start listening before the click…
      await this.addButton.click();
      await saved;                                            // ← …and wait until the server has it
    }
  }
}

The test now reads like the story it's checking:

import { test, expect } from "@playwright/test";
import { ShopPage } from "./shop-page.js";

test("buying two ducks costs $10", async ({ page }) => {
  const shop = new ShopPage(page);
  await shop.signIn("ana@example.com", "s3cret");             // (§08: where this user comes from)
  await shop.addToCart(2);
  await shop.payButton.click();
  await expect(shop.status).toHaveText(/confirmed: \$10$/);   // ← the assertion stays in the test
});

Notice addToCart waits until the server has stored each click before going on. That is the fix for the real failure in §09; because it lives in the page object, every test gets it for free. It waits for the server's answer rather than for the number on screen, so it works whatever the cart held before, even for a shared account whose cart isn't empty (§08).

Rules that keep page objects useful

• Keep assertions in the test, not in the page object. A reader should see what's being checked without opening another file. (Waiting inside a method, like addToCart does, is fine; that's how the page works, not what the test checks.)
• Name methods after what a user does, like signIn, addToCart or checkout, not after the elements they touch (clickButton3).
• One page object per page or big component. A CheckoutPage, a Header, a CartDrawer. Don't build one giant App class.
• Expose locators for anything a test may want to assert on (shop.status), so tests can still use web-first assertions.

§07 removes the last bit of repetition: every test still writes new ShopPage(page) and signs in by hand. Fixtures can hand the test a ready-made, signed-in shop.

Fixtures and isolation

What { page } really is

You've been writing async ({ page, request }) => … since §03. Those names are fixturesFixture (Playwright)A named helper, such as page, request or your own shop, that Playwright builds before a test, passes into the test function by name, and cleans up afterwards. You ask for one just by naming it: async ({ page }) => …. Playwright reads which ones your test asks for and builds only those. The main built-ins:

That "one test" column is the most important design choice in Playwright. Because every test gets a new context, a test that signs in can't leave another test signed in, and tests can run in parallel in any order. Starting a context is cheap: 7 ms in our trace on this machine, compared with ~60 ms to launch the browser itself.

Building your own

Every test in §06 still started with new ShopPage(page) and a sign-in. Move that into fixtures:

import { test as base, expect } from "@playwright/test";
import { ShopPage } from "./shop-page.js";

export const test = base.extend({
  user: async ({ request }, use) => {
    const user = { email: `u${crypto.randomUUID()}@example.com`, password: "s3cret" };
    await expect(await request.post("/test/users", { data: user })).toBeOK(); // §08
    await use(user);                                     // ← the test runs here
  },
  shop: async ({ page, user }, use) => {                 // ← fixtures can use other fixtures
    const shop = new ShopPage(page);
    await shop.signIn(user.email, user.password);
    await use(shop);
  },
});
export { expect };

import { test, expect } from "./fixtures.js";            // ← our test, not Playwright's

test("buying two ducks costs $10", async ({ shop }) => {
  await shop.addToCart(2);
  await shop.payButton.click();
  await expect(shop.status).toHaveText(/confirmed: \$10$/);
});

Step through what happens when that three-line test runs:

1export const test = base.extend({
2  user: async ({ request }, use) => {
3    const user = { email: `u${crypto.randomUUID()}@example.com`, password: "s3cret" };
4    await expect(await request.post("/test/users", { data: user })).toBeOK();
5    await use(user);                      // ← the test runs here
6  },
7  shop: async ({ page, user }, use) => {
8    const shop = new ShopPage(page);
9    await shop.signIn(user.email, user.password);
10    await use(shop);
11  },
12});
13 
14test("buying two ducks costs $10", async ({ shop }) => {
15  await shop.addToCart(2);
16  await shop.payButton.click();
17  await expect(shop.status).toHaveText(/confirmed: \$10$/);
18});

Fixtures vs beforeEach

You can also write test.beforeEach(async ({ page }) => { … }) to run setup before every test in a file. That works, but fixtures have three advantages: they run only for tests that ask for them, they combine (shop builds on user), and the cleanup sits right next to the setup instead of in a separate afterEach.

Expensive things: once per worker

Some setup is too slow to repeat for every test, such as creating an account with a lot of seed data. A fixture can be built once per worker and shared by the tests that worker runs:

export const test = base.extend({
  account: [async ({ playwright }, use, workerInfo) => {
    // `request` is per test, so a worker fixture can't use it: make our own API client.
    const api = await playwright.request.newContext({ baseURL: workerInfo.project.use.baseURL });
    const email = `worker${workerInfo.workerIndex}-${Date.now()}@example.com`;
    const account = { email, password: "s3cret" };
    await expect(await api.post("/test/users", { data: account })).toBeOK();
    await use(account);                                  // every test in this worker shares it
    await api.dispose();                                 // cleanup when the worker shuts down
  }, { scope: "worker" }],                               // ← built once per worker
});

A worker fixture may only depend on other worker fixtures. It lives longer than any single test, so it can't borrow something that is thrown away after each test. The obvious first try, async ({ request }, use, workerInfo), stops the run before a single test starts. Real output:

worker fixture "account" cannot depend on a test fixture "request" defined in <builtin>.

We ran the version above: two tests in one worker both signed in with the same worker account and passed.

Test data and logging in once

Rule 1: seed through an API, not the UI

If a test needs a signed-up user, don't click through the sign-up form to get one. That's slow, and it means a bug in sign-up fails every test, including ones that have nothing to do with sign-up. Instead, call a test-only endpoint (or write to the database directly) before the browser does anything. Putting known data in place like this is called seedingSeeding (test data)Putting known data into the app before a test — a user, a product, an order — usually by calling an API or writing to the database directly, rather than clicking through the UI..

The Tiny Shop's server only registers its seeding route when it's started with APP_ENV=test:

if os.Getenv("APP_ENV") == "test" { // ← test-only routes never exist in production
    mux.HandleFunc("POST /test/users", func(w http.ResponseWriter, r *http.Request) {
        var in struct{ Email, Password string }
        if err := json.NewDecoder(r.Body).Decode(&in); err != nil || in.Email == "" || in.Password == "" {
            http.Error(w, "need email and password", http.StatusBadRequest)
            return
        }
        db.mu.Lock()
        db.users[in.Email] = in.Password
        db.mu.Unlock()
        w.WriteHeader(http.StatusCreated)
    })
}

We checked both: with APP_ENV=test the route answers 201 Created. Without it, Node answers 404, and Go answers 405 Method Not Allowed (the route doesn't exist; the only pattern matching that path is the page's GET /). Either way, production can't be seeded by strangers.

Rule 2: every test owns its data

Each test in this topic creates its own user with a unique email (ana+${Date.now()}@… or crypto.randomUUID()). That's not fussiness. If two tests share "ana@example.com" and run at the same time, one adds a duck while the other is checking that the cart is empty, and you get a failure that only happens sometimes: a Flaky testFlaky testA test that sometimes passes and sometimes fails without any code change — usually because it depends on the real clock, randomness, timing or the network..

• Create, don't assume. Never rely on "there's always a user called admin in staging". Someone will delete it.
• Make it unique. Random suffixes on emails, names and order numbers mean tests can't collide, even across parallel runs or repeated runs against the same server.
• Clean up, or throw it all away. Either delete what you made in the fixture's cleanup (after use(), §07), or start every CI run with a brand-new database so leftovers don't matter. Integration Testing covers database isolation in depth.

Rule 3: sign in once, reuse the cookies

Signing in through the form in every test wastes time and repeats the same steps everywhere. Playwright can sign in once, save the browser's cookies to a file, and start every test from that file. The file is called Storage stateStorage stateA JSON file holding a browser context's cookies and localStorage. Saving it after logging in once, then loading it into every test, lets tests start already logged in..

It takes a setup project: a project that runs before the others, which list it under dependencies:

projects: [
  { name: "setup", testMatch: /.*\.setup\.js/ },                  // ← runs first
  {
    name: "chromium",
    use: { ...devices["Desktop Chrome"], storageState: "playwright/.auth/user.json" },
    dependencies: ["setup"],                                       // ← waits for setup
  },
],

import { test as setup, expect } from "@playwright/test";

const authFile = "playwright/.auth/user.json";

setup("sign in once", async ({ page, request }) => {
  await request.post("/test/users", { data: { email: "ana@example.com", password: "s3cret" } });
  await page.goto("/");
  await page.getByLabel("Email").fill("ana@example.com");
  await page.getByLabel("Password").fill("s3cret");
  await page.getByRole("button", { name: "Sign in" }).click();
  await expect(page.getByRole("button", { name: "Add to cart" })).toBeVisible(); // ← really signed in?
  await page.context().storageState({ path: authFile });                      // ← save cookies to a file
});

Every test in the chromium project now starts already signed in. Real run on this machine:

Running 3 tests using 1 worker

  ✓  1 [setup] › tests/auth.setup.js:5:6 › sign in once (233ms)
  ✓  2 [chromium] › tests/cart.spec.js:3:5 › starts signed in (131ms)
  ✓  3 [chromium] › tests/cart.spec.js:8:5 › can pay (1.0s)

  3 passed (2.7s)

The saved file is just the session cookie, as JSONJSONJavaScript Object Notation: a text format for data — {"id": 7, "tags": ["a"]}. Readable by humans and every language; the default body format for web APIs.:

{
  "cookies": [
    { "name": "sid", "value": "b0beb9db7c9e09e4d25387cb5defcaf3", "domain": "localhost",
      "path": "/", "expires": -1, "httpOnly": true, "secure": false, "sameSite": "Lax" }
  ],
  "origins": []
}

Even faster: skip the browser and sign in over HTTP. await request.post("/api/login", { data: … }) followed by await request.storageState({ path: authFile }) saves the same cookie. On this machine that setup took 40 ms instead of 233 ms. Keep one test that signs in through the real form, so the form itself stays covered.

When it fails: traces and screenshots

What to record

Recording costs time and disk, so the usual setup records only when something went wrong:

use: {
  trace: "on-first-retry",        // record a trace when a failed test is retried
  screenshot: "only-on-failure",  // a PNG of the page at the moment it failed
  video: "retain-on-failure",     // a video, kept only for failed tests
},
retries: process.env.CI ? 2 : 0,  // retry failed tests on CI only

on-first-retry means: run normally; if the test fails, run it again with recording. You get a trace for exactly the tests that failed, without slowing every run down. Locally you usually have no retriesTest retryRunning a failed test again automatically. If it passes the second time, the run is green but the test is reported as flaky. Retries hide flakiness unless someone looks at that report., so add --trace on when you want one. retain-on-failure is the other common choice: record every test, keep the file only if it failed.

A real failure

This test (written without the page object's waiting from §06) checks that two ducks cost $10:

const add = page.getByRole("button", { name: "Add to cart" });
await add.click();
await add.click();                                          // ← no wait for the cart to update
await page.getByRole("button", { name: "Pay" }).click();
await expect(page.getByRole("status")).toHaveText(/confirmed: \$10$/);

It passes on a laptop. To imitate a slow CI machine, we used Network interceptionNetwork interceptionLetting the test catch the page's HTTP requests and decide what happens: delay them, change them or answer them itself. In Playwright that's page.route(). Useful to simulate slow or failing servers.: page.route("**/api/cart", …) held the second cart request back for 300 ms before letting it through:

let n = 0;
await page.route("**/api/cart", async (route) => {
  if (++n === 2) await new Promise((r) => setTimeout(r, 300)); // hold the 2nd request for 300 ms…
  await route.continue();                                      // …then send it on, unchanged
});

The real result:

Error: expect(locator).toHaveText(expected) failed

Locator: getByRole('status')
Expected pattern: /confirmed: \$10$/
Received string:  "Order 6e19fe09 confirmed: $5"
Timeout: 5000ms

  attachment #3: trace (application/zip)
  test-results/race-buying-two-ducks-costs-10-chromium/trace.zip
  Usage:
      npx playwright show-trace test-results/race-buying-two-ducks-costs-10-chromium/trace.zip

"$5 instead of $10" says what. It doesn't say why: did the second click miss? Did the server add wrong? Open the trace. Below is that exact trace, rebuilt from its trace.zip:

The trace makes the cause plain. Both clicks worked; Playwright did nothing wrong. But a click only starts a request. The test clicked Pay at ~404 ms, while the second cart request was still on its way (398–706 ms). The server worked out the total from a cart holding one duck. This is a timing bug in the test: it moved on before the state it depended on was ready.

The fix is one line: await expect(page.getByTestId("cart-count")).toHaveText("2") after the second click. With it, the same test passed with the 300 ms delay still in place. addToCart() in §06 fixes it a slightly different way, by waiting for each /api/cart response, and also passed with the delay in place.

A debugging routine

1. Read the error and call log first. Often enough: "strict mode violation", "element is not visible", "Received: Paying…".
2. Look at the failure screenshot. Is the page even what you expected? An error banner, a login form or a cookie popup explains a lot of failures at a glance.
3. Open the trace and walk backwards from the failed step. Check the network rows: a 500 from the server, or a request still in flight, is the usual culprit.
4. Reproduce locally: npx playwright test --last-failed --headed. Put await page.pause() in the test to stop there with the Inspector open.
5. If it only fails sometimes, make it fail more often: --repeat-each=20 runs it 20 times, and page.route can slow down the request you suspect. Flaky Tests is all about this step.

Running in CI against a preview build

Why not just test the dev server?

npm run dev starts a Dev serverDev serverThe local web server started by npm run dev. It serves your source files to the browser, transforming each one when it's requested, and pushes updates when you save. that transforms each file when the browser asks for it and swaps code live with Hot module replacementHot module replacement (HMR)When you save a file, the dev server sends the browser just that changed module and swaps it into the running page without a full reload, so the app keeps its state (open dialogs, typed text, React state).. It's built for fast editing, not for being identical to production. What users get is the output of npm run build: bundled by a BundlerBundlerA build tool (Vite/Rollup, esbuild, webpack) that follows your imports and combines your modules and libraries into a few optimized files for the browser., minifiedMinifyRewrite code to be as small as possible without changing what it does: remove spaces and comments, shorten local names (calculateTotal → e), and simplify expressions., with production settings.

Bugs that live only in the build are real: a package that loads in dev but not after bundling, an environment variable missing at build time, a wrong base path. Build Tools §05 lists more. So E2E tests in CI should run against a Preview buildPreview buildThe production build of a frontend (minified, bundled — what users will get), served locally with a command like vite preview or deployed to a temporary URL for one pull request., in one of two ways:

A. Build and serve it inside the CI job. For a Vite frontend, vite preview serves the built dist/ folder (on port 4173 by default). For a Go server, build the binary and run it. Either way webServer starts it:

webServer: {
  command: "npm run build && npx vite preview --port 4173 --strictPort",
  url: "http://localhost:4173/",
  reuseExistingServer: !process.env.CI,
  timeout: 120_000,                    // ← a production build can take a while
},

B. Point the tests at a deployed preview. Many hosting platforms deploy every pull request to its own temporary URL. Tests can target it instead of starting anything:

const baseURL = process.env.BASE_URL ?? "http://localhost:8080";

export default defineConfig({
  use: { baseURL },
  webServer: process.env.BASE_URL ? undefined : { command: "./shop", cwd: "./app", url: baseURL },
});

Option B tests the real hosting setup too: the CDNCDNContent Delivery Network: servers spread around the world that keep copies of your files close to users, so they're served from a nearby city instead of your origin server. in front of the site, TLSTLSTransport Layer Security: the encryption layer that makes HTTPS secure. It proves the server is who it claims to be and scrambles everything so nobody in between can read or change it. (HTTPS) and redirects. But you share that environment, so seeded data must be unique (§08) and tests must not assume an empty database.

A GitHub Actions workflow

name: e2e
on: [pull_request]

jobs:
  e2e:
    runs-on: ubuntu-latest
    timeout-minutes: 20
    strategy:
      fail-fast: false                     # let every shard finish, even if one fails
      matrix:
        shard: [1/4, 2/4, 3/4, 4/4]
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-go@v7
        with: { go-version: "1.26" }
      - uses: actions/setup-node@v7
        with: { node-version: 22, cache: npm }
      - run: npm ci
      - run: go -C app build -o shop .     # go.mod lives in app/; builds app/shop
      - run: npx playwright install --with-deps chromium
      - run: npx playwright test --shard=${{ matrix.shard }}
        env: { CI: "true" }
      - uses: actions/upload-artifact@v7
        if: ${{ !cancelled() }}            # upload even when tests failed
        with:
          name: playwright-report-${{ strategy.job-index }}
          path: playwright-report/
          retention-days: 14

The uploaded report is a CI artifactCI artifactA file a CI job saves after it finishes — a test report, screenshots, traces — so you can download it from the CI page later, even though the machine that made it is gone.: download it from the run's page and open it with npx playwright show-report. Traces and screenshots (§09) are inside it.

Settings that matter on CI

forbidOnly: !!process.env.CI,          // fail if someone committed test.only(...)
retries: process.env.CI ? 2 : 0,       // retry failures on CI, report them as flaky
workers: process.env.CI ? 2 : undefined,
reporter: process.env.CI ? [["html", { open: "never" }], ["github"]] : "list",
webServer: {
  command: process.env.CI ? "./shop" : "go run .",   // CI runs the binary the workflow built
  cwd: "./app",
  url: "http://localhost:8080/",
  env: { APP_ENV: "test" },
},

• forbidOnly: test.only runs just that one test, which is handy locally and a disaster if committed (the rest of the suite silently stops running). !! turns the variable into a plain true/false.
• retries: a test that fails then passes is reported as flaky, not passed. Retries keep one random blip from blocking a merge; they're not a fix. Someone has to look at the flaky list. --fail-on-flaky-tests makes flaky count as a failure, for teams that want to be strict.
• workers: CI machines are small (often 2–4 CPU cores), and too many parallel browsers make everything slower and flakier. Start low and measure.
• webServer.command: §03 used go run ., which compiles on every start. On CI the workflow builds app/shop once (the go -C app build step: -C app means "run as if inside app/", where go.mod lives), and the tests start that exact binary.
• reporter: html builds the downloadable report; github turns failures into annotations on the pull request's diff.

Sharding: more machines, not more workers

One machine can only run so many browsers at once. Past that, split the suite across machines. Each machine runs one ShardShard (test sharding)One slice of a test suite. With --shard=2/4, a machine runs only the second quarter of the tests, so four CI machines together finish in about a quarter of the time.:

Four shards finish in about a quarter of the time, but each still pays its own setup (checkout, install, build, browser download), so sharding a 3-minute suite gains little. Each shard also produces its own report; to get one combined report, use the blob reporter on the shards and a final job that runs npx playwright merge-reports.

Visual regression, in brief

Every check so far asked "is the right text there?" None of them would notice the Pay button turning white-on-white, or the sidebar sliding over the form. A Visual regression testVisual regression testA test that takes a screenshot of a page or element and compares it pixel by pixel with an approved screenshot (the baseline). It fails when the look changes, even if every text and button is still right. notices exactly that. It's a cousin of the Snapshot testSnapshot testA test that saves a component's or function's output to a file the first time, then fails whenever the output changes until someone approves the new version., comparing pictures instead of text.

How it works in Playwright

test("sign-in form looks right", async ({ page }) => {
  await page.goto("/");
  await expect(page.locator("#login")).toHaveScreenshot("sign-in.png"); // the sign-in <section> from §03
});

First run: there's nothing to compare against, so it fails on purpose and saves the screenshot as the BaselineBaseline (screenshot)The approved, saved screenshot a visual test compares against. It is created on the first run and updated on purpose (npx playwright test --update-snapshots) when a change is intended.:

Error: A snapshot doesn't exist at …/tests/look.spec.js-snapshots/sign-in-chromium-linux.png, writing actual.

You look at the image, and if it's right, commit it. Later runs take a new screenshot and compare. We added a 4 px margin above the Sign in button and ran it again:

Error: expect(locator).toHaveScreenshot(expected) failed
Locator: locator('#login')
  Expected an image 480px by 112px, received 480px by 116px. 329 pixels (ratio 0.01 of all image pixels) are different.

When a change is intended, accept the new look with npx playwright test --update-snapshots, review the changed PNGs in the diff, and commit them.

Play with the comparison

The comparison has two knobs. threshold (0 to 1, default 0.2) is how different two colours must be before a pixel counts as different. maxDiffPixels (or maxDiffPixelRatio) is how many different pixels are allowed before the test fails. The default allows none.

The three problems you'll hit

• Different machines draw differently. Fonts, Anti-aliasingAnti-aliasingSoftening the edges of text and shapes with in-between colours so they look smooth. Different operating systems, GPUs and font versions do it slightly differently, so the same page can differ by a few pixels. and GPUs vary by operating system. That's why the file name above ends in -chromium-linux.png: baselines are per browser and platform. Generate and compare them in one environment, usually the same Docker image CI uses (Playwright publishes official ones), never on your laptop.
• Things that change every run. Clocks, "3 minutes ago", ads, random avatars, carousels. Pass mask: [page.getByTestId("updated-at")] to paint over them, or freeze them (a fixed clock in test mode). Playwright already turns off CSS animations while it takes the screenshot.
• Review fatigue. A one-pixel change to a shared header can change hundreds of baselines. If reviewers start clicking "accept all", the tests have stopped protecting anything.

Case study: this site's own smoke test

What it does

Every topic on Software Atlas, this one included, is checked with npm run smoke -- <topic-id> before it ships. The script, scripts/smoke.mjs in the repo:

1. starts headless Chrome with a debugging port and connects to it over a WebSocket;
2. opens the topic page and polls until the sections have rendered (reloading once if a dev-server hot reload blanked the page);
3. scrolls to the bottom so every lazy widget mounts;
4. clicks every button inside the article except navigation ones, then presses "Next ›" on every step-through until each reaches its end;
5. fails if anything was logged as a console error or warning, including the site's own [Term] missing glossary entry warning.

Here's a real run, measured on this machine:

$ npm run smoke -- channels
topic: channels · sections: 14 · buttons clicked: 49 · stepper steps: 10 · terms: 18
✓ no console errors or warnings

It took about 12.5–13.5 seconds across two runs on this machine.

It isn't Playwright

The script doesn't use Playwright at all. It speaks the Chrome DevTools ProtocolChrome DevTools Protocol (CDP)The message protocol Chrome's DevTools use to control the browser over a WebSocket: navigate, run JavaScript, listen to console messages. Playwright uses it under the hood for Chromium; you can also speak it directly. directly, the same protocol Playwright uses under the hood to drive Chromium, with nothing but Node 22's built-in fetch and WebSocket. The heart of it is two pieces:

ws.onmessage = (m) => {                          // every message Chrome sends us
  const d = JSON.parse(m.data);
  if (d.method === "Runtime.exceptionThrown") problems.push(/* the error */);
  if (d.method === "Runtime.consoleAPICalled" && (d.params.type === "error" || d.params.type === "warning"))
    problems.push(/* the message */);
};

// …then, inside the page:
const bs = [...document.querySelectorAll(".prose button")]
  .filter((b) => !b.disabled && !/Next|Prev|Replay|Pause|Play|Copy/.test(b.textContent));
for (const b of bs) { b.click(); await new Promise((r) => setTimeout(r, 25)); }

That's a perfectly reasonable choice for this job: zero dependencies, no browser download, one file anyone can read. But measure it against what §02–§11 taught.

Where it bends the rules

That fourth row is the big one. Whatever a test uses to decide pass or fail is its Test oracleTest oracleWhatever a test uses to decide pass or fail. For an assertion it's the expected value; for a crawler that just clicks around, it might only be “no errors were logged”.. Here, the oracle is "the console stayed quiet". That catches the site's most common bugs: a widget that throws when clicked, a missing glossary entry, a React key warning. It can't catch a widget that's quietly wrong.

Is it a bad test, then?

No: it's a smoke test, and a good one. Its job is "does every page render and survive being poked?", for every topic (over 70 of them), with one command, in seconds each. Writing real E2E tests with assertions for every widget on every page would cost far more than the bugs it would find. That's §02's rule applied: spend expensive tests where the risk is, and use cheap, broad checks everywhere else. The site pairs it with npm run check, a static check of glossary keys and section references that runs in about a second.

The same idea in Playwright

For comparison, here's a Playwright version we wrote and ran against the same dev server:

import { test, expect } from "@playwright/test";

const topics = (process.env.TOPICS ?? "channels").split(",");

for (const topic of topics) {
  test(`${topic}: renders and survives a click-through`, async ({ page }) => {
    const problems = [];
    page.on("console", (m) => ["error", "warning"].includes(m.type()) && problems.push(m.text()));
    page.on("pageerror", (err) => problems.push(err.message));

    await page.goto(`/#/t/${topic}`);
    const sections = page.locator("section[data-sec]");
    await expect(sections.first()).toBeVisible();            // ← waits for render, no fixed sleep

    // Click every widget button a reader could click (visible, enabled).
    const buttons = page.locator(".prose button").filter({ hasNotText: /Next|Prev|Replay|Pause|Play|Copy/ });
    let clicked = 0;
    for (const b of await buttons.all()) {
      if (await b.isVisible() && await b.isEnabled()) { await b.click(); clicked++; }
    }

    // Step every step-through widget to its last step.
    for (const next of await page.getByRole("button", { name: "Next ›" }).all()) {
      while (await next.isEnabled()) await next.click();
    }

    // Settle: wait until no new console problem has appeared for 500 ms (at most 5 s).
    let seen = -1;
    await expect.poll(() => { const quiet = problems.length === seen; seen = problems.length; return quiet; },
      { intervals: [500], timeout: 5_000 }).toBe(true);

    console.log(`${topic}: clicked ${clicked} buttons`);
    expect(problems).toEqual([]);                             // ← same oracle: a quiet console
  });
}

We ran both, twice each, on the channels topic, on the same (busy, shared) machine:

The click counts are the honest headline. Playwright pressed 15 fewer buttons because it skips any that weren't visible when it got to them, buttons a reader couldn't have pressed at that moment either. smoke.mjs presses them all anyway. The times are close and noisy (on a more heavily loaded run the two took 18.0 s and 18.4 s), so don't read much into them. Real clicks are a little slower each, since every one scrolls into view and runs the checks from §05, and the settle wait adds about half a second.

Two caveats about our version. buttons.all() takes a snapshot of matches by position (nth(0), nth(1), …); if a click adds or removes buttons, later positions point at different buttons than they did. And it is still only a crawler with a quiet-console oracle. What Playwright adds is traces on failure (§09), parallel workers, retries and reports for free. A fair summary: the CDP script is the right tool for a fast, dependency-free crawl; for journeys that must really work, you'd want Playwright.

Staff level: keeping an E2E suite healthy

Measure two numbers, always

The suite's value is "real bugs caught before users saw them". Its cost is mostly two numbers that creep up quietly:

• Wall-clock time per pull request. Track the trend, not just today's figure. When it doubles, developers start skipping or batching pushes, and feedback gets slower for everyone.
• Flake rate. Of the runs where a test failed, how many passed on retry with no code change? With Playwright, every test reported as flaky (failed, then passed on retry) is a data point. Collect them per test from the reports. A test that flakes in 2% of runs is a bug in the test or in the app, and it needs an owner.

Recall the arithmetic from §02: 200 tests that each flake 0.5% of the time make a red build more likely than a green one. Flakiness compounds, so it has to be fixed at the level of each test.

Hermetic first, shared environments second

A hermetic run starts everything it needs inside the CI job: your server, a fresh database, stand-ins for third parties. Nothing outside can make it fail. A shared environment, like a long-lived staging environmentStaging environmentA copy of production — same code, similar machines and data — used to test changes before they reach real users. Load tests usually run here., is more realistic but brings other people's deploys, other teams' data and outages you don't control.

A good default: run the per-PR suite hermetically (§10, option A), and run a small set against staging or a preview deployment where the real infrastructure matters.

Third parties: fake them at the edge

Your checkout probably calls a payment provider, an email service and loads analytics scripts. Don't let your suite's colour depend on their uptime:

• Payments, email, SMS: use the provider's test mode (sandbox), or point the server at a stand-in in test mode. Mocks, Stubs & Fakes §08 shows httptest.Server and MSW for this.
• Scripts the browser loads (analytics, chat widgets, ads): block or stub them with page.route, so a slow third-party script can't hold up your page load:

await page.route(/googletagmanager|hotjar|intercom/, (route) => route.abort()); // ← not our code, not our test

Keep one or two tests (nightly, not per PR) that do hit the real sandbox, so you notice when the provider changes something.

Cross-browser and mobile, on purpose

Each project in the config runs the suite again with different settings, so adding browsers is cheap to write and expensive to run:

projects: [
  { name: "chromium", use: { ...devices["Desktop Chrome"] } },
  { name: "firefox", use: { ...devices["Desktop Firefox"] } },
  { name: "webkit", use: { ...devices["Desktop Safari"] } },      // Safari's engine
  { name: "mobile", use: { ...devices["iPhone 15"] } },           // small screen, touch, mobile user agent string
],

Three browsers triple the time. Run the full matrix nightly and only Chromium per pull request, unless your users skew heavily toward one browser. devices["iPhone 15"] emulates the screen size, touch and User agentUser agentThe text a browser sends with every request to say what it is, e.g. “…iPhone… Safari…”. Sites use it to tell phones from desktops, so device emulation changes it. in WebKit; it isn't a real iPhone, and real-device bugs still need a real device occasionally.

Production: synthetic monitoring

The last step is to run two or three journeys against production itself, every few minutes, and alert the On-callOn-callTaking turns being the engineer who gets woken up (“paged”) by alerts and must respond, usually for a week at a time. engineer when one fails. This is Synthetic monitoringSynthetic monitoringRunning a few scripted user journeys against production on a schedule (say every 5 minutes) and alerting if one fails — an end-to-end test used as a smoke alarm for the live site.: the fire drill held in the real building, on a schedule.

• Use dedicated test accounts, clearly marked, and stay read-only where possible (search, view product, sign in). If you must test a purchase, use a test card or a flag the payment system honours, and clean up.
• Keep them tiny and very stable. A synthetic check that cries wolf gets muted within a week.
• They also catch what no pre-release test can: an expired TLS certificate, a broken CDN config, a misconfigured feature flag.

Pruning

Every test should be able to answer "what bug would this catch, and could a cheaper test catch it?" Review the suite a couple of times a year:

• Tests that are always green, slow, and duplicate an integration test: delete or demote them.
• Long tests that check twenty things: split them along the journey, or move the detail checks down a level.
• Areas of the app that changed shape: update the page objects (§06) rather than patching each test.

Check yourself

Predict the answer before you click. Q1 is covered in §05, Q2 in §04, Q3 in §02, Q4 in §08, Q5 in §11 and Q6 in §09.

await page.getByRole("button", { name: "Save" }).click();
await page.waitForTimeout(1000);
expect(await page.getByRole("status").textContent()).toBe("Saved");

await page.getByRole("button", { name: "Add" }).click();

await expect(page).toHaveScreenshot("home.png"); // no baseline exists yet

Cheat sheet