Which Pa11y Tool Should I Use for My Accessibility Testing Needs?

Updated

Pa11y publishes four free and open source tools, and the right one depends on who is running the test and how often. For a one-off check of a single page, use Pa11y (the command-line interface). For daily automated testing with trend graphs that non-developers can read, use Pa11y Dashboard. If you want to build your own front end or reuse the underlying data, use Pa11y Webservice. And if you need to test a list of pages as part of a continuous integration pipeline, use Pa11y CI.

The four tools at a glance

Tool What it is Best suited to
Pa11y A command-line interface that loads web pages and highlights accessibility issues Running a one-off test against a web page
Pa11y Dashboard A web dashboard that automatically tests web pages for accessibility issues daily Non-developers who want to see how sites perform, with graphs tracking improvements and regressions over time
Pa11y Webservice The JSON-based webservice that backs Pa11y Dashboard Building your own dashboard-like front end, or repurposing the data
Pa11y CI A command-line tool that iterates over a list of web pages and highlights accessibility issues Use in CI

How to choose

Choose Pa11y (CLI) if you want a quick, one-off result

The command-line interface loads a page and reports the accessibility issues it finds. It is described as useful "when you want to run a one-off test against a web page" — so it fits spot checks, verifying a fix, or trying the tool out before committing to anything more automated.

Choose Pa11y Dashboard if the audience is not developers

The Dashboard runs tests automatically every day and presents graphs that help you track improvements and regressions over time. That combination — scheduled testing plus visual history — is what makes it useful for non-developers who need to see how sites perform rather than read raw output.

Choose Pa11y Webservice if you need the data, not the interface

The Webservice is the JSON-based service that backs Pa11y Dashboard. Reach for it when you want to build your own dashboard-like front end or repurpose the data in another system. In other words, it is the option for when the Dashboard's interface is not what you need, but its data model is.

Choose Pa11y CI if testing belongs in your pipeline

Pa11y CI iterates over a list of web pages and highlights accessibility issues, and is explicitly geared towards use in CI. If your goal is to catch accessibility problems automatically as part of your build or deployment process, this is the tool designed for that job.

A quick way to decide

  • One page, one time → Pa11y
  • Many pages, on a schedule, viewed by non-developers → Pa11y Dashboard
  • You want the JSON data or your own front end → Pa11y Webservice
  • Many pages, checked automatically in CI → Pa11y CI

All four are published as free and open source tools. If you want to help develop them, the site points to a contributors guide as the starting point.