“That's not accessible!” and other statements about accessibility

“We're 100% accessible”, some digital products claim. “That solution is inaccessible”, an accessibility specialist might say. These sorts of statements almost suggest that web accessiblity is a binary thing. Is it though?

In this post, I'll talk about why it's most helpful to see a website's accessibility as a continuum (or, you know, multiple continua). Even then, in some contexts, it makes sense to pretend it is binary.

If you find this interesting, but want actionable advice, see also Adrian Roselli's post Things to Do Before Asking “Is This Accessible?”

It is a spectrum

The accessibility of a website is a spectrum, in terms of different disabilities that exist, in terms of timing and in terms of objective claims.

The goal of web accessibility is that people with disabilities can use the web. In other words, it is about people and maximising the portion of people who can use our UI well. There are people who can't move their arms, who use their screen zoomed in, whose vision is blurry, who control their computer with their voice, and so forth. Accessibility is about people with a wide range of disabilities, that sometimes overlap, too. Our UI could be accessible to most or all of these people, or to some, or to none. Most products are accessible to at least some people with disabilities. Many have specific barriers for users from specific groups. Some do really well at continuously identifying barriers and removing them. Some don't.

Second, it is about timing: today all podcasts on our site may be transcribed, tomorrow we may upload an episode without a transcript. Or launch a new campaign that was done by this agency that didn't take all accessibility requirements into account. It happens. Accessibility can't be solved once and then shipped, it's a continuous process of tracking potential barriers and removing them. “That site is accessible” is a statement that changes over time on websites where content changes.

Third, accessibility conformance testing is subjective to some extent. This isn't a bug in accessibility standards, it's more like a most reasonable choice… If we tried hard, we could invent success criteria that can be evaluated with 100% certainty, but then we would need many of them, they might go out of date fast and they might end up a lot less technology agnostic. The subjectivity serves a purpose, but it's there, and again, a reason that a claim like “this is accessible” is hard to make.

So, basically, there is a degree of subjectivity in determining whether something is accessible, because it matters to which user(s), when it is checked and what is checked. For that reason, a statement like “this is accessible” or “this is not accessible” is best taken with a pinch of salt.

Why pretend it's not

A claim of “great accessibility” is subjective, a bit like “great user experience” and “great design”. Especially when you view “meeting WCAG” as a minimum and aim much higher by doing regular user testing and following best practices beyond WCAG (see my other post about using a superset of WCAG). But sometimes it makes sense to try and make formal and objective-like claims about the accessibility of a website or set of websites. To publish reports that say things like “60% of webshops in Germany are inaccessible” or “Only 10% of online banking is accessible”. Those are usually based on automated tests and/or accessibility conformance reports that refer to standards like WCAG.

One example of when it makes sense to pretend accessibility is objective, is the effectiveness of policy. National governments and organisations like the European Commission want to have a more accessible web, they have this as a policy goal. For that to be more than dreams or empty statements, they need to make it practical and tangible. Their method of measuring success is, roughly speaking, to gather accessibility statements and conformance reports. On the one hand, this reduces the experiences of people with disabilities to checking boxes in a standard, on the other hand, this provides insights at scale, while maintaining a reasonably good representation of individual experiences.

WCAG is used as a way to make statements about websites. Combined with a method like WCAG-EM, a detailed process for evaluating conformance published by the W3C, governments can get some level of certainty.

As an example, The Dutch government has a register of over 3500 accessibility conformance statements. They are each about a specific website, to which a rating between A (“fully meets WCAG”) and D (“does not meet WCAG”) is assigned. Rating E means “statement is missing”. Other efforts include AllAble's accessibility statements research, looking at accessibility statements from public sector bodies in the United Kingdom.

Of course, this approach is not perfect. Individual organisations might be using an auditing agency that is biased or not very good at evaluating WCAG. Some might self-evaluate (generally not a good idea). Or a website could have serious accessibility issues that happen to not be captured by WCAG (it happens). But even with some of those caveats in mind, even if the data is not 100% objective (is data ever?), collecting information from a large set of websites is the best a government can do. Regular formal WCAG/ATAG audits is a great thing for companies and organisations to do too, though ideally that strategy is supplemented by regular user tests and review of best practices.

Wrapping up

In summary: yes, measuring web accessibility is somewhat subjective and claims like “X is accessible” or “X is inaccessible” are tricky. If someone makes such statements, grab a pinch of salt! But, having said that, it can be helpful to talk about accessibility as “meets the criteria in this standard” and “does not meet the criteria in this standard”. Governments do this to measure the success of their policies and companies can do it to have some measurement of their own success. That's mostly useful, even though user tests and best practices based on them are even more meaningful.