How to Read Test Results on wpt.fyi

wpt.fyi shows, for each test, how many subtests passed or failed in a given browser run. Read a result as a fraction (passes / total subtests) plus a status label, then check the run's browser version and date before drawing conclusions. Treat small differences and unstable tests as noise rather than proof of a compatibility gap.

What the numbers and labels mean

Each row in the results view corresponds to a test file, and the number shown is aggregated from the individual subtests inside it.

• Pass/fail counts: a result like 8/10 means 8 of 10 subtests passed. A test with 0 subtests is scored as a single pass or fail.
• Status categories: results are grouped into passing, failing, and other states such as timeouts, errors, or "not run." A test that errors or times out is not the same as one that fails an assertion, and the dashboard distinguishes them.
• Partial results: a test can be partially passing. Do not read a non-100% score as "the feature is broken" without checking which subtests failed.

Grouping by browser, engine, and suite

Results are organized by browser product (for example Chrome, Firefox, Safari) and by the underlying engine, and by test suite or directory. When comparing, keep the same suite and directory selected so you are not comparing different sets of tests. A directory-level summary is useful for a quick overview; drill into individual test files before making a claim about a specific feature.

Filtering and comparing across browsers

Use the dashboard's filters to narrow to a browser, a test path, or a label, then switch between browsers to compare the same tests.

1. Select the test suite or directory you care about.
2. Apply a browser filter, or open the comparison view that places browsers side by side.
3. Confirm the browser version and run date shown for each column.
4. Compare the same test rows, not the aggregate totals, when the question is about one feature.

A practical check: if Chrome shows 100% and Firefox shows 60% on the same test file, open the failing subtests to see whether they concern the same behavior or an unrelated part of the file.

Caveats: flaky and unstable tests

Some tests are flaky, meaning they pass and fail across runs without any code change. Before treating a failure as a real interoperability problem:

• Look at results across multiple runs or dates rather than a single run.
• Check whether the test is marked as unstable or expected to fail in the suite's metadata.
• Prefer a consistent pattern across runs over a one-off failure.

For example, if a test fails in one browser on one date but passes on the next run with the same version, treat it as unstable until a repeated pattern appears.

A quick reading checklist

Use wpt.fyi as evidence about test outcomes, and pair it with the browser version and run date before concluding that a feature is or is not interoperable.