Best Practices for Structuring Accessibility Testing: Part 1

The Importance of Accessibility

Accessibility is a mindset and a set of actions—it’s ensuring people of all abilities have an equivalent experience. Equal access is a basic human right. Without accessibility, people with disabilities are excluded from professional and personal life. Accessibility has a tremendous impact on people’s employment, their ability to access healthcare or their own finances. Working with people with disabilities every day here at TPGi, I witness how it affects a person’s mood when they are unable to access an online experience and must ask for assistance from other people. I’ve come to realize that when businesses connect with the broadest possible audience, they not only sell more of their products, they also make better products.

Therefore, web accessibility is the right thing to do for your customers and for your larger business objectives.

Accessibility Testing with ARC vs. Open-Source Tools

If one has not considered accessibility when creating an online experience, the experience is not accessible. Access is not the default, and it does not happen by accident. Online experiences must be tested to ensure that people of all abilities have access. We at TPGi have created an Accessibility Resource Center (ARC). ARC provides the tools, resources, and a foundation of knowledge, KnowledgeBase to plan and execute in every area of online accessibility, including manual and automated testing. We will identify how ARC supports each area of an accessibility program. We work at TPGi, but more than that, we work for the web and we strongly believe that a rising tide lifts all ships. And so, alternatively, we will describe how, to an extent, one could cobble together an accessibility testing program with open-source tools from around the web.

Habits Are More Important Than Goals

When preparing to perform accessibility testing, it’s important to have a practical philosophy. For your organization, testing all of your existing websites at once may not be possible. While it’s not ideal to potentially have barriers on your website, accessibility is an ongoing process and does not have a single delivery date. Accessibility requires planning, monitoring, and a consistent level of ongoing maintenance.

An emphasis on implementing habits helps far more than a one-time goal. A goal could be having zero automatically detected WCAG errors on your homepage. It’s a good goal, but it’s not clear how your organization would reach it. A habit could be each engineer setting aside four hours a week to conduct manual accessibility testing. With steady habits, people learn fast and over time, these habits compound and lead to the achievement of your accessibility goals.

Accessibility Testing Steps

With this testing philosophy in mind, it’s time to break down the steps to success in accessibility testing. Your roadmap will take stock of your organization’s resources and tweak these steps until you have a unique roadmap that can guide your own personal accessibility journey.

The first step is to create an accessibility roadmap which documents your organization’s testing process by mapping specific responsibilities onto each team with a role in delivering your organization’s online experiences.

Pre-Testing – Access to A Complete KnowledgeBase

To perform accessibility testing, you’ll need an accessibility KnowledgeBase, which is essentially an accessibility encyclopedia. An accessibility knowledge base fosters an organization-wide culture of inclusion and establishes consensus (as each person is referring to a shared and agreed-upon resource). TPGi was founded in 2001 and has performed thousands of accessibility reviews on everything from websites to smart TVs to printers; the ARC KnowledgeBase is the accumulation of all this work. With access to ARC KnowledgeBase, organizations can replicate TPGi’s training and testing processes.

A KnowledgeBase must have three parts – how to understand, test, and solve for accessibility:

  1. Understand Accessibility – Each person with a role in delivering an online experience, from design to development, has a role in ensuring accessibility. When designers, developers, and other roles in the production of an application understand the fundamental principles of accessibility, most issues never reach production. Therefore, an organization needs access to role-based accessibility training. ARC: ARC has Tutor, which consists of 15-to-45-minute self-paced, role-based training modules for designers, developers, PDF authors, and HR professionals, all consolidated in one location. Accessibility managers can assign modules designed for each role and using ARC, track whether each person on the account has accessed the modules. (If your organization wants to share ARC trainings with a wider group beyond just ARC users, the modules can be provided as SCORM files, which you can put in your organization’s LMS.) Open-Source: , To provide trainings using publicly available courses, for specific roles, requires using multiple sites and platforms. For example, for developers, a wonderful open-source training option is Google’s Udacity course, “What is Web Accessibility?” For PDF authors, it’s worthwhile to review Adobe’s Accessibility documentation. A downside to this open-source approach is that most trainings online are not strictly role-based and as a result, are more lengthy and cover material that’s out of scope to that role. It’s cumbersome. Moreover, managers can’t easily track progress or completion.
  2. Test For Accessibility – Comprehensive accessibility testing requires manual testing. Manual testing involves checking a site against the prevailing standard for web accessibility, the Web Content Accessibility Guidelines (WCAG) 2.1 Level AA . This standard is made up of 50 guidelines, or in WCAG terms, Success Criteria. With 50 individual tests, each with its own nuances getting started with manual testing can seem overwhelming. ARC: ARC’s KnowledgeBase includes the WCAG Tests module, which includes step-by-step manual testing procedures for the 50 Success Criteria. With these clear steps, people with less accessibility experience can replicate TPGi’s internal testing processes and reliably carry out full WCAG testing. Open-Source: Alternatively, people could go directly to WCAG. WCAG intends for testers to use its How to Meet WCAG 2.1 reference document. Given WCAG’s legal significance, the document uses W3C jargon, which is precise, even bureaucratic. The language is difficult to understand even for reasonably skilled developers. In contrast, ARC’s KnowledgeBase is focused on comprehensibility. WCAG does not exactly provide step-by-step testing procedures. Each guideline has advisory, supporting content such as “Sufficient Techniques”, which more outlines how one might satisfy the guideline, not how to test for the guideline. Applying these WCAG documents to the ever-increasing range of web content is difficult. Each guideline is written broadly enough to apply to a wide range of web content, so each team will interpret the guideline differently, which leads to a lack of uniformity in testing and a huge amount of re-inventing the wheel. Secondly, the supporting documentation does not lay out the practical steps for testing or make any mention of available testing tools.
  3. ARC – ARC’s KnowledgeBase provides recommended techniques, expected behaviors, and accessible code examples for a large range of UI elements and frontend development techniques. The KnowledgeBase even includes iOS and Android patterns. Open-Source: W3C Web Accessibility Initiative (WAI) provides the open-source ARIA Authoring Practice Guide. It’s a great option but it only covers the limited range of UI elements which have universally-agreed upon patterns. This ignores the reality that organizations develop all kinds of custom UI elements which do not fit neatly into established UI patterns. In TPGi’s vast experience, we’ve made features such as text editors, QR code scanners, and mobile check deposits accessible and shared what we’ve learned in our KnowledgeBase recommended techniques.

Screen Reader Testing

When a webpage supports screen readers, it provides a foundation to support a wide range of assistive technologies. For this reason, a large part of accessibility testing involves screen reader compatibility testing.

JAWS is the most popular primary desktop screen reader. Therefore, it’s most important to test for JAWS compatibility.

JAWS has its own scripting language for organizations to ensure that it works smoothly with desktop programs and JAWS provides significant support directly to its users. Because of this robust ecosystem, JAWS is not free; it requires a license, although it does have a limited free mode.

Using that limited free mode to test poses some challenges. Screen readers have a learning curve even when used simply for testing. When someone responsible for performing accessibility testing has not used a screen reader before, there’s a significant amount of time and effort required to start learning basic commands and configuring the screen reader to best support testing. Let’s imagine this in practice. A tester writes an issue indicating that a JAWS announcement is inappropriate. It’s cumbersome to share JAWS’ current announcement. When an engineer makes an update to the codebase to address the issue, it’s difficult for them to provide evidence that the update has solved the issue.

JAWS Inspect solves this problem by creating a text output of JAWS screen reader speech output. This way testers can easily review and share their results. As for engineers, they can update the code and provide a transcript as evidence that the issue has been fixed. One does not have to have a JAWS license to use JAWS Inspect, instead JAWS Inspect requires a subscription.

Color Contrast Testing

TPGi has a free and easy-to-use Colour Contrast Analyzer (CCA) tool. With the CCA, one can enter and compare color values, or to be more precise, enter hexadecimal codes, or use its eyedropper tool to easily grab any two pixels on one’s screen (not just within a webpage) and test for WCAG compliance.

Now that you have a KnowledgeBase and your team is equipped with testing tools and assistive technologies, you’re prepared for accessibility testing. In part two, you will about learn the basics of performing accessibility testing.

Categories: Accessibility Strategy, World of Accessibility

About Aaron Farber

Aaron is the Senior Accessibility Platform Consultant at TPGi. In this role, he supports the ARC Empowerment Program, which trains organizations to plan and execute in every area of accessibility using the ARC Platform. Aaron has supported web accessibility initiatives at organizations ranging from small businesses using WordPress and Shopify to the largest technology companies in the world. Aaron’s goal is to make accessibility easier to understand and into a mindset that people consider for every situation. Aaron is a former web developer and coding bootcamp instructor.