Skip to content

Module 05 / Lesson 01

Use Playwright and Chrome DevTools to verify the browser

Module 05 · Real-World ContextStep 01 of 03

The AI will always tell you the code is ready. Only empirical verification and human judgment decide when to release.

It is Friday afternoon. The AI agent reports: "All changes implemented cleanly, ready for deployment!" But when a real visitor visits the live site, they receive stale cached information. Furthermore, a hidden security regression was introduced in an unreviewed helper function. Shipping without proof turns an AI shortcut into an outage.

Narrated lesson · 1:36
Read the narration transcript

Two people see different results

The type check is green. The operator publishes an update and the status app reports success. In the operator view, the new update is present.

I open the public URL from a separate session. The new text is missing. Before asking the agent to fix a cache, I want one repeatable example that shows both outcomes.

Pick the browser tool by the verb

Use Playwright MCP to act like a user. Start with an isolated test profile, publish from a seeded operator account, and read the result from a separate visitor session.

Use Chrome DevTools MCP to diagnose. Check the response behind the old incident message and inspect console or layout evidence. Use a test browser profile and disable the tool’s usage statistics if your workflow requires it.

Once the fix is understood, turn the steps into a normal Playwright test. The next reviewer can run the same check with a real assertion.

Save the observable result as a test

Use a unique update in a seeded incident. Assert that the anonymous visitor sees it. Use an assertion that waits for the condition rather than sleeping for five seconds.

Playwright can run WebKit. If your actual issue is specific to an iPhone Safari version, confirm on that browser before closing the report. State which environment actually passed.

Make the regression repeatable

On the old behavior, the visitor assertion should fail. With the intended fix and the same fixture, it should pass.

Keep the test and report the public route and expected result. The next reviewer can now verify the behavior instead of trusting the original walkthrough.

The everyday problem

The status app’s type check is green. An operator can publish an update. Several seconds later a visitor still sees an older status. It’s tempting to guess that publishing failed or that the browser cached the page. Start by reproducing the discrepancy. Where do you first see the saved text, and where do you first see the old text?

Think of this as two people and one release-critical promise. The operator saves through the app. A separate public session reads the status. One browser tab with the operator’s login doesn’t represent both people. Set up synthetic data and note which session is doing each step.

Why the agent got it wrong

A type check can tell you whether code agrees with its types. It can’t tell you whether a published update reaches a browser. Browser automation can reproduce a user journey, but a click-by-click report can stop short of showing why behavior diverged. Look at the evidence between actions: the saved result, the request that reads the data, and the response or cache behavior the visitor actually got.

A diagnosis tool can observe the browser’s network or console. Which request content and timing would change your mind? If the public response carries the new text and the DOM shows the old text, your question shifts to rendering. If the response already contains old text, investigate the data source or cache behavior.

The tool and technique

Pick a tool by the action. Use Playwright or Playwright MCP to navigate like a user, perform an action, and assert the result. Use Chrome DevTools or its MCP integration when you need to inspect network, console, or performance details in Chrome. Ordinary DevTools and a written test can cover the same workflow without giving an agent control of a browser.

Once the behavior and intended fix are clear, save a normal Playwright test. Use meaningful roles and names where they fit. Have the assertion wait for the expected condition instead of adding a blind timeout. A fixed wait can mask the timing issue on one machine and create a flaky test on another.

Use Playwright for the journey and DevTools for diagnosis

Use Playwright MCP to repeat the operator and visitor journey against the local demo. Use Chrome DevTools MCP to inspect the stale response or a layout shift. Follow the Superpowers systematic debugging skill: record what failed, test one explanation against the evidence, then change the cause. Save the verified behavior as an ordinary Playwright test for CI.

npx @playwright/mcp@latest --isolated --browser webkit
npx chrome-devtools-mcp@latest --isolated --no-usage-statistics

Operator publishes “Recovery in progress”.
Independent visitor reloads the public status page.
Assert visitor sees that exact update.
npx playwright test tests/incident-publish.spec.ts

Use seeded test accounts and separate browser profiles. Check that the saved assertion fails with the stale response and passes with the fix. WebKit evidence belongs alongside device-specific Safari checks where required.

On screen in the status app

The first example below waits five seconds and takes a screenshot. You still don't know whether the published message appeared. The revised excerpt takes the read side in a public context and asserts that the publication appears. Use unique update text in each run so a pre-existing message can’t satisfy the test. Seed the incident in the approved fixture rather than relying on state from previous runs.

During the demo, compare the operator response with the visitor session’s request. Once you have fixed the appropriate refresh or revalidation behavior, run the same public assertion again. Keep the assertion visible when presenting the result: the expected content, which user sees it, and which route they loaded.

tests/incident-publish.spec.ts

Before

await page.getByRole('button', { name: 'Publish incident update' }).click()
await page.waitForTimeout(5000)
await page.screenshot()

After

await operator.getByRole('button', { name: 'Publish incident update' }).click()
await expect(operator.getByText('Update published')).toBeVisible()
await visitor.goto(publicIncidentPath)
await expect(visitor.getByText(uniqueUpdateText, { exact: true })).toBeVisible()

What to watch for

Use dedicated test profiles with sample accounts. Browser tooling can see page content and account state. Playwright’s WebKit build is useful for early compatibility checks, but confirm device-specific Safari reports on the actual browser or device before claiming parity. For MCP settings, use the project documentation and check the effective profile rather than assuming a flag exists across all tools.

A single successful test is useful evidence for that behavior and environment. Run it repeatedly enough to catch a quick timing issue, isolate the data, and avoid claiming broad compatibility from one browser. Save logs and traces only as appropriate for the development environment.

How you know it worked

Re-run the original operator-to-visitor sequence using the same fixture. Confirm that the test fails when the application serves the older public version and passes when the intended data is visible. Check that an anonymous visitor is the one making the assertion. Capture the observed result and the coverage limit for the reviewer.

Add the regression to the ordinary automated test suite and keep the report next to the pull request. The next lesson compares evidence from a browser and tests with evidence from static rules and a second review.

In the test report, preserve the visitor URL and the unique update text, without including account secrets. A failure should show which assertion timed out and which session was reading. That makes the report useful to a teammate who never watched the original browser walkthrough.

References