Skip to main content

References

White paper: conducting user evaluations with people with disabilities


IBM internal best practices

Human Ability and Accessibility Center
The IBM Human Ability and Accessibility Center has conducted both in-house and remote evaluations with external and internal visually-impaired, hearing-impaired, and cognitive-impaired users as well as seniors over the last 18 years for their award-winning assistive technologies and clinical products such as IBM Home Page Reader, IBM SpeechViewer, IBM Screen Reader for OS/2, and IBM THINKable. Participants were involved with early design focus groups, remote beta tests, and in-house as well as remote usability evaluation sessions observed by usability experts and the product developers. Participants included seniors and blind, low-vision, and hearing-impaired users from the NFB, AFB, schools for the blind and the deaf, other assistive technology corporations such as Alva who makes Braille displays, large corporations with vision-impaired employees such as State Farm, SeniorNet, speech therapy clinics and public schools associated with the American Speech and Hearing Association, and IBM offices from all over the world.

Silicon Valley Laboratory
The accessibility team at Silicon Valley Laboratory (SVL) has conducted multiple usability sessions with visually impaired users of their Information Management (IM) products. Issues investigated have included the accessibility and usability of IM documentation, the Eclipse-based IM Information Center, and the graphical user interfaces for some popular database administration tools. Results and recommendations from these sessions have directly enhanced the usability of their product deliverables and have been surfaced to the corporate level in some instances to be shared with the rest of IBM's Software Group. This team is also working to integrate testing for accessibility into the formal testing process at their lab, an effort they have deemed Accessibility Verification Testing (AVT).

Hursley Laboratory
The User Experience team at Hursley incorporated users who were blind and visually impaired during their redesign of the IBM syntax diagram standard used in IBM information deliverables. The team sought early design feedback from both internal and external participants and ran sessions throughout the design process at their lab to ensure an accessible and usable final product. The result was the now widely accepted and implemented dotted decimal format for syntax diagrams--a format that has been validated repeatedly as accessible and usable by visually impaired users.

Accessibility Research at TJ Watson Research Center
The Watson Accessibility Research Group has conducted both in house and remote evaluations with local users with disabilities. Most recently, an e-mail and telephone-based study of a keyboard configuration utility was conducted with individuals with young onset Parkinson's Disease across the US. The team at Watson has been leading the way at IBM in user evaluations with users who have multiple disabilities by conducting sessions that have incorporated users with mild to severe motor impairments and aging users. Specific lab-based projects have included keyboard users with moderate motor impairments, people with high spinal cord injuries, people who are deaf, and senior citizens who had mild to moderate motor and visual impairments, often in combination. Most often the setting was a third party location familiar to the participants, such as a senior center, community center or a school. They are continuing to do research on the most effective ways to perform early prototype evaluations with users with disabilities.

BluePages Redesign
The IBM Global Services User-Centered Design Services (UCDS) group had users with disabilities in mind when evaluating IBM's internal BluePages application. Remote user sessions were conducted with IBMers who were blind. These users were sent URLs and tasks ahead of time by the UCDS team. Two rounds of evaluations were conducted during the development and evaluation phases. The evaluation administrator followed along on his own workstation using Home Page Reader while listening to the participant's screen reader output over the phone. Participants were also asked to "think aloud" to further facilitate data collection. While several significant changes were made to the BluePages design following these sessions to improve the usability specifically for by people with disabilities, the changes made likely improved the usability for all users.

External best practices

Nielsen/Norman Group
The Nielsen/Norman Group has done extensive work with users with disabilities, including low vision, blind, hard of hearing, and motor impaired. Their white-papers outlining the process and procedures they followed during these sessions, as well as their results and recommendations, are considered required reading for anyone preparing to run their own user studies with special populations.

ITTATC - Georgia Institute of Technology
The Information Technology Technical Assistance and Training Center (ITTATC) provides accessibility training and technical assistance related to Section 508 of the Rehabilitation Act and Section 255 of the Telecommunications Act. Their audiences include industry, state officials, trainers, and consumers. The ITTATC's goal is to help these groups understand the requirements of Sections 508 and 255 and to achieve success in their efforts to develop, market, and buy accessible technology.

The ITTATC is funded by the National Institute on Disability and Rehabilitation Research (NIDRR) and is located at the Georgia Institute of Technology in Atlanta, Georgia. More information about their work can be found on the ITTATC Web site.

Stanford University Archimedes Project
The goal of The Archimedes Project is to improve the ability of all people to access and use computers and information technologies, regardless of their abilities, needs, preferences, and culture. The Archimedes Project consists of a group of individuals committed to this goal. They are a diverse group with multiple academic disciplines and real-world experience. This includes scholars, researchers, and engineers from both public and private sectors. Their collaborative projects are worldwide, including Ritsumeiken University and Waseda University (Japan), Royal College of Art (UK), Trinity College (Ireland), and Auckland University of Technology (New Zealand).

The Archimedes Project identifies barriers to universal access to computers and information and they design and implement solutions for overcoming those barriers. They identify the technologies and applications that solve specific problems and that improve access to information and computer usability.

The Archimedes Access Research and Technology International, Inc. (AARTI) was established as a non-profit corporation. They design, develop, and test prototypes incorporating technologies originating from Archimedes Project research. A for-profit AARTI Holdings was created to license the prototypes to companies worldwide to manufacture and market prototype-based products. For more information, visit the Stanford Archimedes Project homepage.

University of Wisconsin Trace Center
The Trace Research & Development Center is a part of the College of Engineering, University of Wisconsin-Madison. The Trace Center Mission Statement is "To prevent the barriers and capitalize on the opportunities presented by current and emerging information and telecommunication technologies, in order to create a world that is as accessible and usable as possible for as many people as possible."

In order to design for the general population, it is important to understand the diversity, problems, tools, and abilities of its members The Trace Center has adapted a universal design which includes the general population, those with and without disabilities. Examples of universal design solutions include: people in a noisy shopping mall who cannot hear a kiosk, people who are driving their car and must operate their radio without looking at it, people who left their glasses at home, people who are getting older, and people with disabilities.

The Trace Center does not sell products. The Trace Center works to encourage companies to make their standard products more accessible and usable by people with disabilities of all types. Strategies and techniques developed at the Center are sometimes patented and available via licensing through the Wisconsin Alumni Research Foundation.

Descriptions of major project areas may be found on the web pages for the Center's major programs, as follows:

  • Information Technology Access RERC
  • Telecommunication Access RERC
  • Universal Design/Disability Access Program (NCSA)
  • Universal Design Research Project