pa11y.org

What is Pa11y?

Pa11y is a family of free, open-source accessibility testing tools. Its stated goal is to help designers and developers find and fix accessibility issues on web pages, on the premise that a more accessible web works better for everyone. It is not a single app but a set of related projects, each aimed at a different workflow.

The four tools

How to choose

If you need a quick answer about one page, start with the CLI. If you want a shared, always-current view that colleagues who don't use a terminal can read, the Dashboard is the natural fit. If your goal is to stop accessibility regressions before they ship, Pa11y CI is the one to wire into your pipeline. The Webservice matters mainly if you want to repurpose results rather than use the supplied interface.

A practical next step: pick a small set of representative pages — home, a form, a checkout or signup flow — and run them through the CLI first. That gives you a baseline before you commit to daily monitoring or CI integration. For broader context on the standards these tools test against, the W3C publishes the Web Content Accessibility Guidelines.

One caveat worth keeping in mind: automated tools catch a useful subset of issues, not everything. They are a good safety net and a way to track trends, but they don't replace manual testing or input from people who use assistive technology.

How do I run a one-off accessibility test on a web page using Pa11y?

Pa11y's command-line interface (CLI) is the tool for a one-off accessibility test: you point it at a single URL, it loads the page, and it reports the accessibility issues it finds. That makes it a good fit for spot-checking a page you have just changed, or for a quick audit before a release, without setting up any ongoing monitoring.

A typical first run

1. Install the CLI (Pa11y's own documentation and contributors guide cover installation and usage; the project publishes its tools as free and open source).
2. Run it against one page, for example a command in the form pa11y https://example.com/page.
3. Read the report in your terminal. Each issue usually identifies the page, the type of problem, and the element involved.
4. Fix the most severe items first, then re-run the same command to confirm the change.

What it is good at, and what it is not

• Good for: a single page, a one-off check, reproducing a problem a user reported, or verifying a fix.
• Less suited to: watching a whole site over time. For that, Pa11y offers a Dashboard that tests pages daily and graphs improvements and regressions, aimed at people who are not developers. Pa11y CI iterates over a list of pages and is geared towards continuous integration, so it fits teams that want the check to run automatically on every change. The JSON-based Webservice backs the Dashboard and is useful if you want to build your own front end or reuse the data.

A practical next step

Pick one representative page — ideally a template used across your site — and run the CLI against it. Treat the output as a starting point rather than a verdict: automated tools catch many issues but not all, so combine the results with keyboard and screen-reader testing on the pages that matter most. If the one-off run proves useful, the natural progression is to move the same URL list into Pa11y CI so the check happens on every change, and to use the Dashboard if you need to show progress to people outside the development team.

How can I set up automated daily accessibility testing for multiple pages with Pa11y Dashboard?

Pa11y Dashboard is the piece of the Pa11y family built for exactly this: it automatically tests web pages for accessibility issues daily and graphs the results so you can track improvements and regressions over time. It is aimed at people who want ongoing monitoring rather than a single check, and the project notes it is especially useful for non-developers who need to see how sites perform.

A practical setup path

1. Pick the pages that matter. Choose a representative set — homepage, key landing pages, a form page, a checkout or contact page — rather than every URL on the site.
2. Install and run the dashboard. Pa11y Dashboard is open source, so you host it yourself; follow the project's own installation instructions for your environment.
3. Add your URLs to the dashboard and let the scheduled daily run do the work.
4. Review the graphs, not just the latest score. The value is in spotting a regression after a deploy versus a slow improvement over weeks.
5. Route findings to whoever fixes them. The dashboard is the detection layer; someone still has to triage and change the code or content.

Choosing between the Pa11y tools

A concrete scenario

A small charity with a five-page website and no dedicated developer could install the dashboard, list those five pages, and check the graphs once a week. When a content editor publishes a new page with an unlabelled image or a low-contrast button, the daily run surfaces it before it becomes a pattern across the site.

Decision criterion

If you need a scheduled, multi-page view that non-developers can read, the dashboard is the right fit. If you want accessibility checked on every pull request instead, Pa11y CI belongs in your pipeline — and the two can coexist.

Next step: start with the official project at Pa11y to confirm current installation requirements, then add your highest-traffic pages first.

How do I integrate Pa11y CI into my continuous integration pipeline?

Pa11y CI is the command-line tool in the Pa11y family built for running accessibility checks across a list of pages, which makes it the right member to wire into a build pipeline. The general pattern is: install it, point it at a list of URLs or sitemap, run it as a build step, and treat a non-zero exit code as a failed build. Pa11y's own page describes Pa11y CI as iterating over a list of web pages and highlighting accessibility issues, geared toward CI use.

A practical integration shape

1. Add it to your project so the version is pinned and reproducible, then invoke it from your pipeline script rather than relying on a globally installed binary.
2. Create a config file listing the pages to test and the standard to test against. A JSON config with a urls array and a standard value is the usual starting point; check the project's documentation for the exact option names and accepted standard values rather than guessing.
3. Run it after deployment to a test or preview environment, not against source files. Accessibility testing needs rendered pages, so the site must be reachable.
4. Fail the build on errors. Pa11y CI exits non-zero when it finds issues, which is what lets your pipeline block a merge or release. Decide deliberately whether warnings should also fail; many teams start by failing only on errors to avoid an immediate wall of red.
5. Publish the output as a build artifact or job log so failures are diagnosable without re-running locally.

Thresholds and the first-run problem

An existing site usually fails on day one. Two workable approaches: set a failure threshold so the build only breaks above a known count, then ratchet it down over time; or scope the initial run to a small set of high-traffic templates and expand. The first keeps the pipeline honest while you work through the backlog. The second gives faster feedback and avoids a config file nobody wants to maintain.

Where the other Pa11y tools fit

A common setup pairs Pa11y CI for merge-blocking checks with the Dashboard for trend visibility, since the Dashboard is described as useful for non-developers to see how sites perform over time. If your team wants accessibility numbers in an existing internal tool, the Webservice is the backing JSON API for that.

A concrete scenario

A three-person team deploying a marketing site weekly adds Pa11y CI as a post-deploy step against the staging URL, with the config listing the homepage, a product page and a form page. Errors fail the build; the full report is uploaded as an artifact. Monthly, they lower the threshold as fixes land. That gives fast, specific feedback on the pages most likely to matter without asking anyone to interpret a raw report mid-release.

Next step: read the config and CLI documentation on Pa11y to confirm current option names, then run it once locally against your staging URL before adding it to the pipeline. If you want the results visible to people who do not read build logs, look at Pa11y Dashboard on the same site.

Can I build a custom accessibility dashboard using the Pa11y Webservice?

Yes. The Pa11y Webservice is the JSON-based service that backs the Pa11y Dashboard, and the project explicitly describes it as useful for building your own dashboard-like front end or repurposing the data. So a custom accessibility dashboard is a supported use case, not a workaround.

What that means in practice

You run the webservice yourself, point it at the pages you care about, and read its JSON output from your own front end. Your dashboard controls the presentation: which sites and URLs appear, how results are grouped, how trends are charted, and who on the team sees what. Pa11y supplies the accessibility results; you supply the interface and the workflow around it.

A concrete scenario: an agency with 30 client sites wants one internal dashboard showing each client's accessibility status, with results filtered by client and a weekly email summary. The webservice provides the data layer; the agency's developers build the views, filters and notifications that Pa11y Dashboard doesn't offer.

Decide between the two paths

A sensible next step

Start with Pa11y Dashboard to confirm the tests and page list are useful, then move to the webservice when you hit a limit the dashboard can't solve — custom grouping, embedding results in an existing internal tool, or feeding accessibility data into another system. If your real goal is automated checks on every code change rather than a dashboard, Pa11y CI is the better fit; it iterates over a list of pages and is geared towards CI use.

The project publishes these as free and open source tools, and it invites contributors through its contributors guide if you want to extend or improve them. For related tooling from the same community, see Pa11y.

How can I contribute to the Pa11y open source project?

Pa11y's own site points contributors to a "contributors guide" as the starting point for getting involved. The project describes itself as publishing free and open source accessibility tools, so contributions generally fall into the same categories as any open source developer tool: code, documentation, bug reports and testing.

Where contributions fit

• Code and fixes: The project maintains several distinct tools — Pa11y, Pa11y Dashboard, Pa11y Webservice and Pa11y CI — so a contribution usually targets one specific repository rather than "Pa11y" as a whole.
• Documentation: Each tool has its own audience (CLI users, dashboard users, CI users), so clearer setup and usage docs are a realistic first contribution.
• Issue reporting: Because the tools test live web pages, reporting a page that produces a false positive or a missed issue is genuinely useful feedback.
• Testing and triage: Trying the tools against your own sites and reporting what breaks helps maintainers prioritise.

A practical next step

Start with the contributors guide linked from the Pa11y homepage, then pick the single repository most relevant to your skills. For a first contribution, a documentation fix or a well-reproduced bug report is usually lower-risk than a code change to the testing engine.

Choosing where to help

For broader context on the accessibility rules these tools check against, the W3C publishes the underlying guidelines, and GitHub is where most open source contribution workflows — issues, pull requests, contributor guides — actually happen.