How Pa11y Fits Into a Continuous Integration Workflow

Pa11y CI is the tool in the Pa11y family built for continuous integration: it is a command-line tool that iterates over a list of web pages and highlights accessibility issues, so you can run it as a repeatable step in a pipeline. If you want scheduled testing with a visual history instead of a per-build gate, Pa11y Dashboard is the better fit. If you want to feed results into your own system, Pa11y Webservice exposes the underlying data as JSON.

The four Pa11y tools and where each one belongs

Pa11y publishes a range of free and open source tools, each aimed at a different point in the testing process:

The distinction that matters for CI is between Pa11y and Pa11y CI. Plain Pa11y is described as useful "when you want to run a one-off test against a web page." Pa11y CI is the one "more geared towards use in CI" because it works from a list of pages rather than a single URL.

How Pa11y CI works as a pipeline step

The mechanism is straightforward: you give Pa11y CI a list of web pages, it visits each one, and it highlights the accessibility issues it finds. That maps cleanly onto a CI job, where the same set of pages is checked on every run.

The practical shape of the workflow is:

1. Define the page list. Collect the URLs you want covered — typically the key templates or routes of your site rather than every possible URL.
2. Run Pa11y CI as a build step. Because it iterates over the list, one invocation covers the whole set.
3. Read the reported issues. Each page's accessibility problems are surfaced in the output, so a failing run tells you which page and which issue.
4. Decide the gate. Whether a finding should fail the build or just be reported is a policy choice for your team; Pa11y CI's job is to produce the findings consistently.

The value in CI comes from repetition: the same list is checked the same way on every run, so a regression introduced by a change shows up as a new issue rather than being discovered later by hand.

When to use the Dashboard or Webservice instead

Pa11y CI is not the only way to automate this, and the site is explicit about who the other tools serve.

Pa11y Dashboard automatically tests web pages daily and graphs the results so you can track improvements and regressions over time. The site describes it as "useful for non-developers to see how your sites perform." If your goal is ongoing visibility for a wider team rather than a build gate, this is the intended tool.

Pa11y Webservice is the JSON-based service that backs the Dashboard. It is the right choice "if you want to build your own dashboard-like front end, or repurpose the data" — for example, pushing accessibility results into an existing internal reporting system.

A common arrangement is to use them together: Pa11y CI catches issues at the point of change, while the Dashboard and Webservice provide the longer-term trend view.

What to check before you commit to this setup

• Confirm the tool, not just the brand. "Pa11y" alone is the one-off CLI; CI runs need Pa11y CI specifically.
• Scope your page list deliberately. Pa11y CI iterates over a list you supply, so coverage is exactly what you put in it — nothing is crawled automatically on your behalf.
• Separate the two questions. "Did this change introduce an issue?" is a CI question. "Is our accessibility improving over time?" is a Dashboard question. Picking the wrong tool for the question is the most likely source of friction.
• Check the projects directly for current setup details. The Pa11y site points to its open source projects and a contributors guide; installation and configuration specifics live with each project rather than on the landing page.

If you want to help develop these tools, the site directs you to the contributors guide to get started.