Usability testing is an important part of any website, application, or self-service device quality assurance process. It helps companies understand how people are interacting with the digital asset and gives UX, design, and development teams valuable insight on what could use improvement.
Usability testing can also provide key data about the accessibility of your digital assets. When done correctly, testing can reveal pain points experienced by users with disabilities. It can also help support teams identify and prioritize improvements needed to make a digital asset more accessible and more usable for all.
In this post, we’ll explore how usability testing for accessibility is conducted and how the information gathered can be implemented into your website and digital products.
What is usability testing?
Usability testing is a term that’s long been used across industries to refer to the process of determining how well a digital product, such as a mobile app, self-service kiosk, or website, supports users as they perform tasks and seek to achieve goals. The data collected identifies ways to improve the product.
Usability testing analyzes the paths people take to complete a task, the effort required, and if the user is successful. The goal is to find out how intuitive and easy-to-use the product is for target users, understand user behaviors, and to identify problems users encounter that could be addressed through design changes.
When people with disabilities are involved in usability testing, product teams can gain additional insights into the impact of accessibility barriers on the ability of users to independently and efficiently complete tasks.
Usability vs. accessibility testing
Usability testing and accessibility testing are sometimes used interchangeably, but there are differences.
Usability testing measures the overall effectiveness of a product, focusing on user interface interactions and the extent to which tasks can be completed successfully, with minimal effort or disruption. The purpose of usability testing is to understand how real users interact with a website, mobile app, or self-service device and determine which features to add or improve.
The purpose of digital accessibility testing is to understand how well a digital product conforms to accessibility standards, using standard test procedures to test pages, screens, and functionality for potential accessibility barriers. Accessibility testing is effective in identifying code or visual design changes necessary to remove barriers, but because it focuses on testing against standards rather than observing people with disabilities interacting with the product, accessibility testing doesn’t provide the task-based user focus that usability testing enables.
Why do we need usability testing?
When a company designs a new website or creates a digital product, the design and development team may make assumptions about how the product will be used. For example, using certain colors to identify different steps a user should take or designing interactions in a preferred way.
These choices are likely made for a reason–they make sense to the company and the people working on the project. However, once implemented, the resulting product may not be easy to use by the target audience, including people with disabilities.
Usability testing puts the website, application or self-service device into the hands of target users. When you include people with disabilities in usability testing efforts, you can identify areas where the product could be improved for people who use a range of assistive technology, and where it currently works well.
Conducting usability testing with people with disabilities before a website or digital product launch, and ideally regularly throughout the product development lifecycle, ensures that once people begin using it, it will be accessible and most importantly usable to all.
How to conduct usability studies with people with disabilities
The first step in usability testing is to establish what you want to learn and how you can design a study to gather data to give you useful insights. This includes:
- Establish the scope of the test. Do you want to dive in to evaluate the usability of a specific piece of functionality you’ve built, or conduct a broader evaluation of how easy it is to complete one or more tasks? This will be influenced by who’s sponsoring the study, as well as the time and budget you have to conduct the study.
- Define who you want to participate in the study. Is the study also involving non-disabled participants? If so, make sure recruitment identifies some people with disabilities too. If your study is focused on people with a disability, think about the disability profiles you want to cover. For example, you might want to focus on a specific accessibility profile, like screen reader users. Or you might want to cover a broader set of disabilities, such as blind and low vision, motor disability, people with learning/reading disabilities, and people with limited attention/memory.
- Identify one or more representative tasks that test participants should perform. Think about any expected behaviors for successfully completing the task, if these are known.
- Decide what task success looks like, such as completing in a certain amount of time or within a certain number of errors.
The next step is deciding how to design and run the usability test.
Types of usability testing
In-person vs. remote testing: Will you bring users to a specific location to test the product or will this be done at their home or workplace? In-person usability testing for accessibility gathers richer data as you can directly observe users’ behavior, but is more expensive to run and requires more effort for the test participants. In-person testing may be the only option for products like self-service kiosks where testing needs to be conducted on a specific device in a specific location.
Remote testing, where you and test participants are not together in the same location, is usually cheaper to perform, and gives everyone more flexibility for scheduling, as travel is not required and participants can take part in their preferred location using their own devices. Although you may not get as much data as from direct observation of participants, remote testing can still generate valuable data on potential issues.
Synchronous vs. Asynchronous: Will you be observing and evaluating the user’s experience as they work? Or will you ask users to record data as they perform tasks unobserved? Synchronous testing is moderated by the product or web design team in real-time which lets you answer user questions or ask your own questions of the user based on your observations of their behavior.
Asynchronous gives testers the freedom to evaluate the product when it’s best for them and without the pressure of knowing a team is watching. For asynchronous testing, you’ll have to provide users with a way to record and share test data that’s reliable and useful to you.
What data to collect: Usability testing can be used to gather quantitative and qualitative data. Quantitative data, such as error rates, success rates and completion time can be used for benchmarking and comparison, supporting data-driven decisions.
Quantitative data includes descriptions of behaviors and reactions to the experience of trying to complete tasks and reflections and explanations of approaches. This information helps you better understand what users did, what influenced their approach, and how you might design to better accommodate these behaviors.
Usually, you’ll want to capture a mix of quantitative and qualitative data in usability testing. What’s important is that you make sure the data you gather is as useful and reliable as possible, so think carefully about the questions you ask and how participants might respond.
Of course, testing is not the last step; it’s crucial to assess the information gathered to identify how to make improvements and retest if needed. Teams should look at usability testing for accessibility as an iterative process, where improvements are made, and the testing process is repeated until goals are met.
When you should conduct usability testing
At a minimum, usability testing with people with disabilities should happen before a website or digital product is released to the public, at a point where there is time to make essential changes before launch. Preferably, usability testing should take place early and throughout the product development lifecycle of designs and functional code so that any usability issues are identified and addressed as early as possible.
As companies’ accessibility programs mature, they should incorporate regular usability tests as part of their overall accessibility program and review process. At TPGi, we offer a unique AT User Flow Testing service that is designed for agile teams and provides usability feedback from people using assistive technology quickly so you can resolve accessibility barriers and continue with your sprint.
Regardless of what stage you are at it’s important to ensure your digital products can be used by people with disabilities. If it’s not usable, it’s not accessible.
Want to discuss usability testing with people with disabilities for your next project? Speak with an accessibility expert today.