This week, I’m doing an accessibility audit for a client. One of the first steps is to have a general look at the site. You can – and should – do that manually for sure, but another very useful way to get a good first impression of how good or bad things are is to use an automated tool like Pa11y.

Pa11y is a set of free and open-source tools with which you can test a website for all kinds of accessibility (a11y) issues like insufficient color contrast, missing alt texts, missing labels, wrong heading levels, problems regarding ARIA roles, or general HTML validation issues. There are different tools available: pa11y itself is a command-line interface to run a one-off accessibility test against a web page. Pa11y Dashboard is a web interface for Pa11y to constantly keep an eye on the current a11y status of one or more websites. It is powered by Pa11y Webservice. And then, there is Pa11y CI, an accessibility test runner built using Pa11y that lets you run tests as part of a Continuous Integration environment. This is especially useful if you want to make sure that no issues make it to the live site.

Installing Pa11y #

Node.js is a prerequisite – you might want to install it via nvm first. Then, install Pa11y globally via the command line:

npm install -g pa11y

Now, you can run tests against a URL…

pa11y https://example.com

…or a file:

pa11y ./path/to/your/file.html

Pa11y will run the test and display the results in the terminal.

There is much more you can do with Pa11y, in particular the JavaScript Interface, but this would be beyond the scope of this post. So have a look at the GitHub repository in case you want to learn more.

Pa11y CI #

Running a test against a URL or a file is already super helpful, but wouldn’t it be nice to run a test for a whole site and, let’s say, get a report of the results in HTML?

This is can be done with Pa11y CI. While Pa11y CI is best used to run automated tests, you can run it manually as a command-line tool as well. And it comes with a neat feature: you can tell it to run a test against a whole sitemap. Let’s try this. First, we install Pa11y CI globally:

npm install -g pa11y-ci

Now, you could create a config file that includes all the URLs that Pa11y CI should check. But you can also feed it with an XML sitemap with the --sitemap option:

pa11y-ci --sitemap https://pa11y.org/sitemap.xml

Pick Your Reporter #

We’ve seen that when we run a test with either pa11y or pa11y-ci, the results will be output to the command line by default. But this can be changed by switching the so-called “reporter” from cli to something else. You could use csvor json, for example, but you can also output the test results as HTML. Pa11y now has an HTML reporter built in. The reporter can be set via the --reporter option. So when you are using Pa11y, you can write:

pa11y --reporter html https://example.com

But in our case, we want to run a test against a whole sitemap with Pa11y CI. Here, we have to install one additional rpm package that adds an HTML reporter: Pa11y CI HTML Reporter.

You install it via npm again:

npm install pa11y-ci-reporter-html

Now, this reporter can again be set via the --reporter option:

pa11y-ci --reporter=pa11y-ci-reporter-html https://pa11y.org/

Pa11y CI will now run its test(s) and output the results in the current working directory as HTML. Nice! In the example above, we ran the test against a single URL, so the last step is to run the tests against a sitemap using the HTML reporter:

pa11y-ci --reporter=pa11y-ci-reporter-html --sitemap https://pa11y.org/sitemap.xml

The result:

Et voilà! We have generated a nice HTML overview of the test results of the whole website we want to test. Now, we can go on and dig deeper, document existing issues in more detail, or even – if we have access to the codebase – immediately fix a few things to reduce the number of issues we see.

Again, an automated tool like Pa11y is not a substitute for a proper accessibility audit by an actual human being – particularly one using a screen reader. But it does give you a good general overview and helps you to quickly address many of the things that are wrong in your codebase.

~