Usability testing is an important part of any quality assurance (QA) process when designing and building websites, applications, and other digital resources. Effective usability testing can validate whether people can use functionality as intended and can reveal critical barriers in user journeys.
When people with disabilities are included in usability testing, usability issues can be found that might not be caught through accessibility testing. By understanding how users interact with your digital products, you can find issues and make necessary improvements to optimize the user experience (UX) for everyone, including disabled people.
In this post, we’ll explore usability testing, how it contributes to accessibility efforts, and how you can improve the usability of your website and digital products, such as a website, mobile app, or self-service kiosk.
What Is Usability Testing?
Usability testing is the process of evaluating how well a digital product supports users in performing tasks and achieving their goals. This process provides valuable data on how intuitive and user-friendly the product is, and highlights areas for improvement.
Usability testing focuses on key tasks that a user of the product would expect to be able to complete independently. It may involve measuring:
- The approach users take to complete a range of tasks.
- The time and effort needed to complete those tasks.
- The success rate of task completion.
Usability testing is best conducted with people who are representatives of the product’s target audience but who are not members of the product team. The goal of this testing is to understand user behaviors, find barriers to ease of use, and identify design changes that will reduce barriers and enhance usability. To do that, always prioritize learning how users behave when using your product over asking them what they think about it.
Usability Testing vs. Accessibility Testing
Usability testing and accessibility testing are distinct but complementary practices:
Accessibility Testing
- Focus: Assesses how well a product conforms to a standard such as WCAG (Web Content Accessibility Guidelines) using standard test procedures.
- Purpose: To find accessibility barriers through technical inspection of code, content, functionality, and appearance. Accessibility testing does not have the task-based user focus that usability testing provides.
Usability Testing
- Focus: Observe representative users interact with a product by performing representative tasks.
- Purpose: To understand real users’ interactions with the product, find barriers to easy task completion, and find features for improving usability.
Both practices are essential for developing digital assets that are both technically accessible and user-friendly for people with disabilities. While accessibility testing helps find issues that should be addressed before conducting usability testing with people with disabilities.
Usability testing with people with disabilities helps you verify that your technical accessibility efforts have led to a product that can be successfully and independently used. In addition, it helps you understand where issues exist and how disabled users are affected so you can improve the digital product in ways that will specifically benefit disabled users.
For a deeper dive into accessible user experience (UX), refer to our comprehensive series on UX and accessibility strategy.
Types of Usability Testing Methods and Data
Different methods can offer valuable insights:
In-Person vs. Remote User Testing
- In-person testing gathers richer data, as you can directly observe users’ behavior. However, it is more expensive to run and requires more effort for the test participants. In-person testing may be the only choice for products like self-service kiosks, where testing needs to be conducted on a specific device in a specific location.
- Remote testing allows you and test participants to not be together in the same location. It is usually cheaper to perform and gives everyone more flexibility for scheduling, as travel is unnecessary. Although you may not get as much data as from direct observation of participants, remote testing can still generate valuable data on potential issues.
Synchronous vs. Asynchronous User Testing
- Synchronous testing is moderated by the product or web design team in real-time. This lets you answer user questions or ask your own questions of the user based on your observations of their behavior.
- Asynchronous testing gives testers the freedom to evaluate the product when it’s best for them and without the pressure of knowing a team is watching. For asynchronous testing, you’ll have to provide users with a way to record and share test data that’s reliable and useful to you.
Quantitative vs. Qualitative User Testing Data
- Quantitative data includes metrics like error rates, success rates, and completion time and can be used for benchmarking and comparison, supporting data-driven decisions.
- Qualitative data provides insights into user behaviors and experiences, helping to understand the context of their actions. This information helps you better understand what users did, what influenced their approach, and how you might design to better accommodate these behaviors.
Generally, it’s helpful to collect a mix of both quantitative data and qualitative data. That way, you can understand what you need to focus on to improve usability, while also supporting benchmarking performance against a baseline target.
How to Conduct Usability Testing
The first step in usability testing is to establish what you want to learn so that you can design a study to gather data to give you the insights you need. With that objective in mind, you can then:
- Establish scope: Plan out what you’ll be testing. You might want to evaluate specific functionality or conduct a broader assessment. This depends on the study’s goals, timing in the project lifecycle, budget, and time constraints.
- Define who to include in your testing: Based on goals and budget, you should define how many participants to include. The more people you recruit, the richer the data. But that will also take more time and resources. When you include people with disabilities, consider whether to recruit across a broad range of disabilities or focus on one specific disability group.
- Select representative tasks: Choose realistic tasks that participants would be expected to be able to perform when using the functionality being tested. Don’t be too ambitious with the number of tasks and make sure there’s time to ask participants questions afterwards about their experience
- Design and run the usability test: Choose whether to run the test remotely or in-person. Make sure participants are briefed on what they will be doing in advance. Compensate participants for their time. Watch what they do, try not to interrupt or help them if they get stuck, and ask them to explain their approach if they did something you didn’t expect. Ultimately, focus on gathering information that will help you improve the product user experience.
When Should I Conduct a Usability Test?
When a company creates a new website, app, or other digital product, the design and development team may make assumptions about how the product will be used. For example, the product team might use certain colors to find different steps a user should take or design interactions in a preferred way.
These choices were likely made because they made sense to the company and the people working on the project. However, once implemented, the resulting product may not be easy to use for the target audience, including people with disabilities.
Ideally, usability testing should occur early and throughout the product development lifecycle. Regular testing as part of your accessibility program helps spot issues at the earliest possible stage, reduces the risk of expensive and technically challenging remediation work later in the development lifecycle, and helps ensures your digital product is usable by people with disabilities.
As your company’s accessibility programs mature, incorporate regular usability testing as part of your overall accessibility program and review process. At TPGi, we offer a unique AT User Flow Testing service designed for agile teams and provides usability feedback from people using assistive technology quickly, so you can resolve accessibility barriers early and effectively.