Our world is becoming increasingly intelligent, interconnected, and instrumented, resulting in massive amounts of data being collected. This data is a treasure trove of information that can be mined to improve service, increase sales, or make operations more efficient. Analysis of such large amounts of data, often called analytics, is increasingly desired by governments and businesses alike. Yet getting useful information from such large amounts of data is a daunting task. IBM understands that analytics often relies on real-time visual renderings that allow users to quickly spot trends and gain insights. These visual renderings tend to be complex charts (bar, line, scatter, or bubble charts, timelines, node diagrams, etc.) or editable node diagrams. Visual charts can be challenging to understand, especially for persons with disabilities.
One of IBM's internal review boards has formed a working group to research these issues and develop strategies to address these needs. This article describes some of the accessibility challenges of charts, large datasets, and node diagrams and some techniques to make them more accessible and usable by people with disabilities. These techniques can be used to make products with complex charts accessible.
Accessibility standards, such as W3C Web Content Accessibility Guidelines (WCAG) 2.0, guideline 1.1, (link resides outside of ibm.com) require alternative text descriptions for all charts. In order to be a meaningful replacement for the chart, the alternative text should communicate the insight that can be gleaned by someone who can see it. Examples of such insights might include general trends, trouble spots, largest categories, highest and lowest values, etc.
For example, Figure 1 is a bubble chart illustrating secondary education levels segmented by gender starting in 1970 and at four additional points in time ending in 2008.
On a static information website or report, the author creates the alternative text for the chart based on his or her judgment as to what information is important given the context of how the chart is being used. For the chart in Figure 1, an author might summarize in alternative text as follows:
Data from 5 points in time from 1970 through 2008 shows the largest number of degrees awarded is bachelor’s degrees and increasing trends at all degree levels in both the total number of degrees and the percentage of women receiving secondary degrees.
However, when a chart is created real-time from data that changes dynamically, there is no opportunity for someone to analyze the chart and create alternative text that communicates the message of the chart. Adding to the complexity of this scenario, the context or use of the chart may not be known. A software analytics tool may not know the user’s goals for examining the dataset. In this case, best practices, such as the WGBH Science, Technology, Engineering, Math (STEM) Guidelines, (link resides outside of ibm.com) then advise providing the data itself in an accessible structure such as a table. While certainly possible, wading through a table with thousands of rows or columns of information does not afford a comparable experience to the visual experience of using a chart. After all, the reason the visual renderings are created is to make it easier to spot trends, identify problem areas, and gain insights from the large amount of data that must be analyzed. Charts are a compact way to highlight relationships among the data points in the set such as differences, similarities, connections, and patterns. They are an important way to provide data for those who have language disabilities.
Adding complexity to this scenario, users consuming the chart may want to manipulate it to discover new insights for themselves. This means being able to reorder, group, combine, filter, sub-classify, simplify and normalize the data. Results must be able to be compared in many different manners. In order to make these charts or large datasets accessible to users with disabilities, mechanisms are needed to help them get a high level view and quickly drill down under their own control to areas of interest.
Accessibility Principles for Alternative Renderings
In looking at different types of complex visualizations implemented in IBM products, we have developed the following high level accessibility principles for providing an alternative view of the data itself.
Pie & Bar Charts, Scatter Plots, Line Graphs
Tables are often provided when all users need access to the data or as an alternative to pie and bar charts, scatter plots, and line graphs. But even accessible tables may be challenging for users with disabilities. For example, Figure 2 illustrates a fragment of a much larger table of sales data.
To make the data easier to consume, some kind of chart, such as a bar chart, might be provided. Users who can't see the bar chart will have a hard time gaining the same insight from the very large table of data that sighted users get from the bar chart.
In this example of a large, complex data table with large numbers, the usability of the table can be improved by providing filters, interactive queries, and scaling of numbers.
Filters can be provided to display only particular rows or columns or to exclude particular rows or columns such as the example in Figure 3. The table has been filtered down to display only the total product sales in 2007 for each type of sales method offered. The figure also illustrates an example of the Cognos Business Intelligence product user interface for applying filters to tables. This technique allows the user to narrow down the data being examined.
For sorted data, providing the ability to mask out rows, displaying only every nth row is another technique that can be useful. Users can then scan the data quickly to get to an area of interest where the masked rows can then be redisplayed for more detailed analysis.
In addition to filters, interactive queries are also very helpful in improving accessibility and usability of large tables. Queries are highly context dependent so it’s difficult to give specific rules for them. Some examples of queries that would be useful are:
Figure 4 is an example user interface from an IBM Cognos product that provides a way to query for the top or bottom values.
Scaling the numbers
When the numbers in tables are very large, a sighted user will scan visually and subconsciously filter out the less significant digits. But when listening to the numbers with a screen reader, it becomes difficult to distinguish the differences as the user navigates from cell to cell. Providing a mechanism to scale the numbers to fewer significant digits mimics the technique that sighted users employ naturally.
Figure 5 provides an example. After the scaling factor is applied, the numbers in the table are much simpler to digest aurally.
Tagging is a feature supported on many Websites that allows users to add their own tags to pages to help find related content. A tag cloud is provided on the Website displaying the tags that users have added. The most frequently used tags are displayed side by side in a small, compact area. The number of tags displayed may be limited by the amount of space allocated to the tag cloud or there may be a user setting (typically a slider) to display more or fewer tags. Each tag name is a link to a list of pages that have been given that tag.
The tags are typically styled differently depending on the number of pages on the site that have been given that particular tag. More frequently used tags are larger, bolder, and perhaps a darker color than less frequently used tags. This technique is useful for sighted users because it draws their eye to more frequently used tags. It is not accessible to blind users, however, because it is conveying information through presentation that cannot be determined programmatically. In other words, it does not meet WCAG 2.0, guideline 1.3.1 (link resides outside of ibm.com).
Techniques to make tag clouds more usable and accessible include:
Figure 6 illustrates an alternative view of a tag cloud where the tags are listed alphabetically with each tag's frequency of use on the site.
A word cloud, sometimes called a Wordle, is a way of showing the frequency of word use in a document. Similar to a tag cloud, the words appearing more frequently in the document are styled with larger, heavier fonts and darker colors. And rather than being text links arranged side by side in a compact space, the words are displayed in a horizontal and vertical arrangement. In addition to the styling differences, words occurring more frequently are arranged towards the center while those occurring less frequently are arranged towards the ends.
An interactive text view, such as that developed for the IBM Research project Many Eyes, is a good way of providing an alternative rendering for word clouds. Figure 7 shows an example.
The text view provides a high level summary, a set of properties that can be modified by the user, and a detailed table.
The Summary section provides the total number of words in the document or dataset and the number of words being displayed. It also provides small tables listing the top 5 and bottom 5 words.
The Properties section provides user interface controls for modifying the word case, layout preference, number of words to display, and filtering out specific words or words based on language properties such as common English words.
The Details section provides a simple data table displaying the words according to the user’s preferences: alphabetically or by frequency of use, in ascending or descending order. If the table is too large to display in its entirety, paging controls are provided.
Another example from the Many Eyes project is illustrated in Figure 8. A bubble chart on the right is a good way of comparing a particular characteristic or metric of a collection of items. In this particular rendering, the bubble chart is displaying state population size estimates for 2005. The size of the circle is directly proportional to the population estimate for that state. Looking at the chart, a sighted user can quickly see that California and Texas are the two most populous states followed by New York and Florida. Using the interactive text view on the left, blind users can obtain the same information by examining the table in the Detailed Data section. The user can select the census data metric he or she would like to examine and the desired sort order.
The interactive text view is also useful to users who are unable to use a mouse or pointing device. For the smaller states, where the circle is too small to include the name and population of the state, the only way to see the information is to position the mouse pointer on the circle. A hover bubble will appear displaying the name and population estimate of the state that corresponds to that circle. Users who are unable to use a mouse can obtain this information using the table in the Detailed Data section in the text view. The bubble chart also allows the user to select circles using a mouse in order to obtain a total sum of the population numbers in those states. The text view provides this function for users who are unable to use a mouse via the Selected checkbox in the Detailed Data table. The number of items selected, their total and a small table containing only the selected items is displayed in the Summary section of the text view.
By allowing the user to drill down, focus on data of interest, and compare and summarize the data on-the-fly, the text interaction approximates the process available to graph interaction. The user is allowed to seek, sift and compare the data. This allows the user to discover insight on his or her own.
One final example from the Many Eyes project is a histogram. Figure 9 illustrates a histogram rendering using the same census data set as used above in the bubble chart. In this case, rather than total population, the metric being examined is the percentage of the population that is 65 years of age or older in 2004.
In the interactive text view, illustrated in Figure 10, the user has selected 4 states, North Dakota, Maine, Texas, and Washington, to be highlighted in the Summary section. In this text view, an interactive query is also provided. The user can simply enter the name of the desired state and select the Search button to obtain information about that particular state.
Some datasets, such as organization charts or server network infrastructures, lend themselves to representation as node diagrams where entities are displayed in a graphical form and connected to indicate relationships that may have many different meanings including ownership, access, succession, prerequisites, equivalence, etc.
Node diagrams can be quite complex. In the case of mission critical applications like network systems management, they must be editable in a highly efficient manner to correct node failures and traffic bottlenecks to maintain network service levels and reliability. These complex node diagrams are difficult to understand and use efficiently for users who are blind or vision-impaired or who use only the keyboard for interaction. As with other kinds of data, understanding the nature of the complex diagram requires understanding the way the objects are connected, including any meanings associated with the connections. The logical and physical layouts of the objects need identification, as well as the ability to reorganize in different layouts to observe the implications. As with data comprehension, users need to discover their own insights.
This is supported through grouping, filtering, and channeling of the objects. Properties and classification of the objects need to be apparent, and the notation under the control of the user.
The first big problem is that there are no clear established standards for how these diagrams should be navigated using the keyboard. Testing with users found that the user needs to know what the choices are without losing focus on the current object. The link or relationship depicted by the arrow may have information such as costs or limitations associated with it. The link may be as important as the subsequent object at the other end.
Another issue is the semantic identification of the nodes and connections for screen readers. And there may be meaning implied simply by the layout of the node diagram. A common form of these is sometimes called swim lanes where elements are grouped in a horizontal or vertical way to indicate owner relationship, location, or chronological timing.
General Approaches for Node Diagrams
The first approach to node diagrams is the development of a special model for navigation that we call the "steppingstone model." In this model, the focus remains on the current object, and the user is presented with a list of the links, and information about them, which can be selected. The user can listen to and consider each of the potential choices before actually selecting a choice. An important advantage of this approach is the keyboard focus is not lost or transferred in order to discover the choices. This approach reduces problems with the user getting lost in a maze of links that may not have an easy way back. At this time, the steppingstone model is our recommended approach to IBM products developing accessibility for complex node diagrams.
One of our product development teams has proposed and been experimenting with mapping the nodes and their relationships into a tree diagram. This mapping seems to have potential in diagrams that have a limited number of branches, for example organization charts or business processes which diverge through several choices and then come together. The question remains open as to what to do with nodes that join back up. Trees generally converge and expand by nature but do not converge back together. There needs to be a mechanism to "jump" to other points in the tree and communicate the context of the new location. This would probably not be a good model for a network topology type of diagram where the nodes are peers and can be connected to most any other node. However, it may be appropriate for some limited number of hierarchical diagrams such as organizational charts, where nodes are connected only to parents and children.
Time Lines and Gantt Charts
A Gantt chart is a particular type of complex node diagram: a visualization of a project plan and its associated line items. It can have several complexities including multiple inputs and outputs to any particular task in the chart. In addition there needs to be some indication of the critical path – those tasks that determine the shortest path to the completion of the end task. Every task in the system is not going to be part of the critical path or even connected to the main trunk of the project so there has to be some way to discover unconnected tasks. There are also some of the same problems with keyboard navigation as found in other node diagrams. The user needs to understand the choices for navigation from the current node without losing focus. And there are no standards for this kind of keyboard navigation.
The design approach currently being considered is to have three widgets and a toolbar working together as illustrated in Figure 11. Note that this implementation is a Web application. WAI-ARIA (link resides outside of ibm.com) roles, states, and properties are used to provide the semantic information needed by a screen reader.
These three widgets are the Project Tree on the left which lists all tasks in the system. The Project Tree can have nested subtasks within the task. The second is the Cascade View which shows the tasks connected by their dependencies as well as an indicator of the percentage of the task that is complete. And the third is the Resource View which lists the tasks for each individual. It shows "busy bars" for the individual and the tasks over time.
Keyboard operability in the Cascade View uses some of the "steppingstone" solution previously described. As the user moves from the Project Tree to the Cascade View, the focused item is synchronized between the views. In this way the user gets the task details using the Project Tree and uses the Cascade View to navigate between the tasks, following the inter-task relationships. At any time the user can move back to the Project Tree and hear the details associated with that view. In contrast, an element in focus in the Resource View is not synchronized with the Project Tree and Cascade View. The Resource View has its own focus memory when the user returns.
In the visual rendering icons provide an efficient mechanism for communicating the critical path. To provide the same efficiency for blind users, audio tones, or "earcons," are used to indicate the critical path.
To implement a steppingstone model each task has three focus areas: predecessors, the task itself, and successors. The Right and Left arrow keys are used to navigate between the focus areas. The predecessor and successor focus areas have the WAI-ARIA (link resides outside of ibm.com) role of button. When focus moves to a particular task, the successor and predecessor buttons each open a list of links. Up and Down Arrow keys are used to listen to the successor (or predecessor tasks) and then the Enter key is used to select the desired task. Continuing with the right (or left arrow) takes focus to the next (or previous) listed task. The next listed task is the one in the order of the Project Tree, and is not necessarily the successor task. This design allows the user to traverse the entire Cascade View simply with the right arrow key. This also maps to familiar WAI-ARIA navigation models.
More Work to Do
These are a few examples of the best practices we have developed to address complex visualizations. But there are other complex visualizations and more solution ideas to be explored.
Maps are one area that needs further study. Maps are highly context sensitive. It's easy enough to provide turn by turn text directions if the purpose of the map is to illustrate navigation from one point to another. But maps can be used for many things:
These other map contexts need further study.
Automated Importance Recognition
In the future we hope to come up with automated ways to identify the information that is important to the user based on the context of the data. This feature may be more difficult to automate as "importance" is very context dependent. Some examples are identifying:
This is the kind of information that would fit well in the Summary section of the Many Eyes text view in the future.
Dense Data and Aggregations
Figure 12 illustrates an example of highly dense data. It shows a set of temperatures taken throughout the day on each day of the year. The temperatures range from warm in the summer to cool in the winter but generally cooler during night time hours. A pattern emerges from the color coding which shows a common time of coldness in the evening.
To represent this in raw data in a table would be possible. But the sheer magnitude of the data would obscure the meaningful patterns. So, the question is how do we make that meaningful? Perhaps aggregation into blocks of time or quadrants (or more) with data for the aggregate division that can be compared. Perhaps just deviations from average are important. Perhaps there is a way to encode the data in sound. These are areas for future research.
Figure 13 is a frequency distribution of bugs found in a project coded to their severity level. Again this is the place where a table would provide the information but not necessarily the understanding. However, an important part of this display is that the pattern is distinctly cyclic. It has an increasing wave pattern. This is something that would not be evident just from looking at the data. But perhaps a filter or pattern matching tool could be used to match and then describe the pattern.
Figure 14 shows dense data connections between multiple nodes. Again this is a case where a table would provide information but not necessarily understanding.
We need to find ways to identify visual patterns, and repetition in cycles. Perhaps there is something in mapping data to other modes that would help.
Once the data can be analyzed and patterns can be found, the goal would be to provide automated analysis, and then take that automated analysis and provide descriptive text, to somehow find correlations between relationships in the data and then map that correlation or relationship into significant meaning in text for a user.
Perhaps technology similar to that used for Watson, the computer built to play and win Jeopardy! could be applied to solve these problems of complex visualizations. Watson takes sentences, picks them apart, then does pattern matching on those same words to find similar patterns in other textual fragments. These patterns are then sorted for significance, selected for best fit, and then reassembled into a form of a question.
Maybe in the future this technology will have applications in finding patterns in data, patterns in visual data, and recognizing their similarity to other known patterns that could be communicated back to the user through text descriptions.
As is often the case with accessibility, best practices for making complex visualizations accessible have applicability in mainstream solutions as well. As more and more business applications move to mobile platforms, similar accessibility issues will be experienced by all users due to limited screen size, the hardware platform itself, viewing the information outside in the bright sun, and of course the current input mechanisms which simply do not adequately address how different users utilize virtual keyboards or gesture technology.
Accelerating the use of graphics to represent large amounts of data will be innovations in Web technology which are already being implemented. HTML 5 will introduce a new feature called
<canvas> and provide greater support allowing for easier integration of Scaleable Vector Graphics (SVG). The integration of these drawing technologies without the need for a plug-in will allow authors to deliver rich graphics with complete fidelity across browser and operating systems. HTML 5 is already becoming a mainstream technology on mobile devices and given the limited display size will accelerate the need to aggregate large amounts of data into consumable graphics. This will only exacerbate the problem unless good solutions are identified which are accessible. Given the situational disabilities imposed by mobile devices, the need to integrate accessibility technology and strategies will be essential for all users.
Graphic visualizations are a valuable way to understand analytics. Although, of themselves they are not accessible, principles of presentation exist to make the textual form of the data more easily consumed by those with disabilities. Techniques of interaction provide a text-based user an ability to discover data insights in a manner similar to that of graphics viewers. Node-based diagrams can be more easily navigated with consistent navigation models, such as those which have been discussed. In the future aggregation of data and pattern matching may assist in describing complex data.
From any analysis, the future looks bright for persons with disabilities using complex analytics.
About the Authors
Brian Cragun is a Senior Accessibility Consultant with IBM’s Human Ability and Accessibility Center. He consults with Lotus, Rational, Tivoli, IM and Business Analytics brands. He co-chairs the IBM Accessibility Architecture Review Board (AARB)subcommittee on Mobile Accessibility. Brian has a broad background in Graphical User Interface development. He is an active inventor with over 130 filed and 80 issued patents, many in the areas of user experience and accessibility. He has been designated a master inventor, and chairs the Emerging Technology Invention Review Board in Rochester. Brian received his undergraduate degree in Computer Science from Utah State University in 1982 and his Masters in Manufacturing Systems Engineering from University of Wisconsin – Madison in 1986.
Andi Snow-Weaver is currently the worldwide accessibility standards program manager for IBM, co-chair of the IBM AARB, and chair of the AARB Complex Visualizations working group. She was formerly the leader of the CI 162 team and from 2000 until 2009, she was IBM's representative to the W3C Web Content Accessibility Guidelines (WCAG) working group working on WCAG version 2.0. In addition to WCAG, Andi was a member of the Telecommunications and Electronic Information Technology Advisory Committee (TEITAC) developing recommendations to the U.S. Access Board for updating the Section 508 and 255 accessibility standards. She served as co-chair of the TEITAC subcommittee addressing Web, software, and electronic content. And, Andi also drives IBM’s involvement in all international accessibility standards organizations, such as the W3C, ISO, JTC-1, OASIS, JIS, JSA, IMS, and country-specific accessibility policies around the world in support of a harmonized global approach to IT accessibility.
This paper is based on work being done by the Complex Visualizations working group of the IBM Accessibility Architecture Review Board. All members of the group have contributed to developing the best practices described in this article. In particular, the authors would like to acknowledge and thank David Dracoules, James Taylor, Robert Kinsman, Mary Jo Mueller, Christophe Jolif, Stephen Levy, Susann Keohane, David Dewar, and Rich Schwerdtfeger for their contributions to the examples and best practices described here that were developed for a CSUN 2011 session presented on behalf of the team.