TOWARDS A UNIVERSAL COGNITIVE TOOL: DESIGNING ACCESSIBLE VISUALIZATION FOR PEOPLE WITH INTELLECTUAL AND DEVELOPMENTAL DISABILITIES

Keke Wu, University of Colorado Boulder keke.wu@colorado.edu

Abstract

Data visualization leverages human visual system to enhance cognition, it helps a person quickly and accurately see the trends, outliers, and patterns in data. Yet using visualization requires a viewer to read abstract imagery, estimate statistics, and retain information. These processes typically function differently for those with Intellectual and Developmental Disabilities (IDD) and have created an inaccessible barrier for them to access data. Preliminary findings from our graphical perception experiment suggest that people with IDD use different strategies to reason with data and are more sensitive to the design of data visualization compared with non-IDD populations. This article discusses several implications from that study and lays out actionable steps towards turning data visualization into a universal cognitive tool for people with varying cognitive abilities.

Introduction

Data visualization offers a method for seeing the unseen. It is the use of computer-supported, interactive, visual representations of data to amplify cognition [1]. Extensive empirical studies have been done to examine the optimal configuration of visual encodings pertaining to cognitive performance, and the research community has since developed a set of golden design principles [2, 3, 4]. These commonly adopted guidelines, however, were developed without considering the special traits and needs of people with IDD and are failing to meet their growing demands of data analysis [5, 6]. This is particularly problematic in that we are increasingly dependent on data to gain insights, make decisions, participate in society, and interact with one another. Meanwhile, rapidly growing complexity of data and ever-expanding scenarios of data analysis have offered a host of emerging opportunities and challenges [7]. These changes require visualization to make visible the vivid nuances of different types of data, adapt to analysts with varying levels of abilities, and reinvent itself to provide more vibrant forms of cognitive support.

As data visualization is a cognitive tool by nature, I define cognitively accessible visualization as cognitively adaptable graphical representation, and I argue that it should benefit people with varying cognitive abilities in accessing data, including those with IDD. My hope is to modernize the design of visualization and unleash its expressive power to animate the data experience and capture the vibrancy of humanity. Throughout my dissertation, I plan to investigate (1) how people with and/or without IDD interpret common data visualizations differently (Graphical Perception - completed); (2) how people with and/or without IDD approach data and build visual representations (Participatory Design – in progress); (3) how we can support people with and/or without IDD in accessible visual cognition and communication (System Design & Development - planned). This article provides an actionable summary of the findings of a web-based mixed methods experiment on (1) and discusses several open challenges facing the design of accessible visualization with and for people with IDD as planned in (2) and (3).

Preliminary Findings & Actionable Insights

Inspired by an ongoing effort [5] to support people with IDD in their financial self-advocacy and decision making, we collaborated with the Coleman Institute for Cognitive Disabilities to make budgetary data accessible to people with IDD. We identified three visualization design variables— chart type, chart embellishment, and data continuity —and conducted a web-based mixed methods experiment to compare task performances and chart preferences between people with and without IDD [8]. Our results suggested that people with IDD used different strategies in interpreting data and that they were more sensitive to the design of visualization than those without. It is reasonable to conclude that people with IDD can analyse quantitative information and appropriately designed visualization can enhance their access to data. Below, I will discuss the four initial design guidelines (illustrated in Figure 1) and their actionable implications drawn from this experiment.

Figure 1 illustrates the four accessible visualization design guidelines coming from our experiment. The first guideline is to avoid pie charts. We suggest using stacked bar charts instead of pie charts when representing proportion data. The second guideline is to use familiar metaphors. We encourage replacing a traditional line graph with a thickened line with semantically meaningful icons such as dollar signs to help a viewer connect data to meaning. The third guideline is to manage visual complexity. We recommend using visual embellishments in moderation to avoid distracting and overwhelming a viewer. For example, a traditional treemap consisting of regular colored squares is more accessible than a discrete version with unnecessary stick figures on top of it. Lastly, we also suggest the use of discrete encodings for axis-aligned representations. For example, a bar graph, which comes with an x and y axis, is more accessible when we break its bars into countable dots.
Figure 1. Accessible Visualization Design Guidelines for People with IDD from [8]

#1 Avoid Pie Charts

Grounded on cognitive and perceptual theories, data visualizations map data to charts for optimal task performance [3, 4]. For example, bar charts are commonly used to compare data across categories [9], line charts to illustrate the change over time [9], pie charts are considered strong for representing the part of whole relationships in data and so forth [10]. Our experiment, however, challenged these conventional mapping strategies. The quantitative results showed that people with IDD struggled to estimate quantities with pie charts and were more than twice as accurate with stacked bar charts and tree maps. In the qualitative interviews, though, participants preferred pie charts and tree maps over stacked bar charts as they felt that the large, familiarly shaped design may help them better understand the data. While we couldn’t conclude why pie charts were the least accessible given the limited visualizations being tested, it is reasonable to assume that the best mapping between visual variables and data for a given task will differ for people with IDD. In addition, certain visual task such as length comparison (as in reading stacked bar charts) may have better accessibility than others such as area and/or angle estimation (as in reading pie charts). It may also be possible that visual tasks that involve less dimensions lead to better cognitive performance than those involve more. It appears important then to understand how people with IDD approach different types of data and perceive visual attributes and find their association to support optimal task performance.

#2 Use Familiar Metaphors

Morden visualization design maximizes the data-ink ratio and avoids unnecessary decoration (“visual noise”) as much as possible [2]. However, this approach would remove any non-data elements in a graph, deprive data of its context [2], and may add difficulty in understanding visualization. Recent research has showed benefits of chart embellishments on cognitive performance, such as enhanced memorability, comprehension, and recall [11, 12, 13]. Guidelines on web accessibility also suggest providing contextual information for the user to build a better mental model that integrates visuals with associative memory [14]. In our experiment, in the context of budgetary and demographic data analysis, we tested the impact of icons (dollar sign / stick figure) and chart junk (money stack / stick figure as background image) on data interpretation. We found that people with IDD reasoned about data through analogues to real world objects and preferred using chart types which evoked real world shapes. For example, they described bar charts as “rising stairs”, pie charts as “pizza”, and tree maps as “colored papers”, and expressed preferences for bars over lines in estimating trends for the more concrete and “noisy” design. These findings suggest that semantic pictorials may benefit mathematical reasoning [15] and scaffolding visual cognition with familiar metaphors may help a viewer to better understand the information. Since our brain prefers taking in information gradually, and people with IDD benefit from progressive disclosure of information, visualization might present data in a more guiding and revealing manner. As an engaging way of building context and bring data to life, visual storytelling is becoming an emerging direction [16, 17] and may help address the multidimensional challenge in reading data visualization.

#3 Manage Visual Complexity

Despite its substantial benefits in mathematical reasoning and visual communication, we found mixed effects of using embellishment for people with IDD. While visual embellishments could add interest and increase engagement, they could also over¬whelm. And there was a difference within participants with disabilities: people with intellectual disabilities tended to remark positively on embellishments, people with autism, however, tended to prefer abstract visualizations, in line with recommendations for visual simplicity from the W3C [18]. As visual complexity varies from person to person, we cannot make a blanket statement about all people with IDD. For visualizations to be accessible, designers have to put people in the center, considering their unique abilities, design visualizations to be responsive to individual differences, and adaptable to personal needs. Recent efforts in diversifying audiences may also offer implications. Peck et al. [19] interviewed people from rural area and found that data is inherently personal, being able to recollect one’s own experiences and resonate with the messages being conveyed in the visualization can fundamentally shift a viewer's mindset. Roberts et al. [20] found that visualization for personal learning must be engaging, and it has to show a clear benefit to persuade students to put extra effort into storing and manipulating data. Rall et al. [21] worked with human rights experts and identified an interest in using visualization to promote advocacy materials as well as a need for novel forms of spatial visualization. These together, emphasize the need to recognize individual differences, be open to various scenarios of using visualization, and collaborate with different audiences to incorporate personally meaningful, cognitively friendly embellishments to the design of visualization.

#4 Use Discrete Encodings for Axis-Aligned Representations

Prior work demonstrated working memory benefits of “chunking” visual in¬formation [22] such as using isotype visualizations [23]. Our experiment confirmed this and showed that discrete encodings improved accuracy and response times for peo¬ple with IDD, significantly reduced disparities with the control group. In the specific case of bar charts, participants with IDD noted a strong preference for discrete representations in that being able to count the points in bars with close values helped them com¬pare values that were further apart and made them more confident in their response. When representations were not axis-aligned, as in examples of pie charts and tree maps, this discretization, however, seemed to overencourage counting and led participants to second-guess themselves. Intensified visual clutter may in addition lead to distraction and cause unintentional stress and anxiety in participants with IDD. Besides, we also noticed a potential of using discrete encodings for novel purposes. For example, while scatter plots are not used as commonly as line charts to represent the trend of change, we found that participants had overall better performance with these discrete representations. This in part may be due to the need to “mentally connect the dots” when using the scatter plot for trend estimation, in which case it can potentially induce active thinking in the viewer to draw a conclusion. It also highlights the possibility of repurposing conventional visualizations and reconfiguring visual encodings for novel creations. In fact, appropriation isn’t new in HCI and user experience research. For example, Hamidi et al. [24] explored DIY assistive technologies (DIY-ATs) with people with and without disabilities through a series of community-building activities. They found that users customized the DIY-AT for specific needs, and that these flexible mechanisms may allow users with disabilities to appropriate similar DIY-ATs in innovative ways in new contexts. As visualization is also a collection of arrangeable visual variables, it would be an interesting future work to allow people with IDD experiment with different configurations and giving them full creative freedom to have fun with data.

These results, in a nutshell, challenged the one-size-fits-all tradition, and have brought to light new demands of understanding individual differences, repurposing old charts for new use, and integrating context into data to design relatable visualizations.

Conclusion

Data visualization connects vision and cognition, it helps a viewer make sense of visual information, and accomplish tasks fast and accurately. Mastering data visualization, however, takes a solid grasp of literacy, numeracy, and graphicacy, which poses an inaccessible barrier to people with IDD. Meanwhile, data visualization is also a bridge between the eyes and the heart, it delivers messages, thoughts, ideas, and emotions through compelling visual communication. Using judicious visual embellishments and storytelling strategies may enhance its communicative power and engage viewers with IDD to participate in data analysis. In this article, we provided an actionable summary of an experiment on how people with and without IDD read common visualizations and several implications on accessible visualization. We believe designing truly accessible visualization will need participatory design and co-design approaches with people within this community, and that’s our critical next step.

Acknowledgments

This work was supported in part by The Administration on Commu¬nity Living DHHS #90DNPA0003-01-00. I would like to thank Dr. Danielle Albers Szafir (my advisor), Dr. Ben Shapiro, and Dr. Emily Shea Tanis for being constant sources of inspiration for this work. Many thanks to ATLAS institute, the VisuaLab, a team of collaborators, and all the participants for their support.

References

  1. Card, S. K., Mackinlay, J. D., & Shneiderman, B. (1999). Readings in information visualization: Using vision to think. San Francisco, Calif: Morgan Kaufmann Publishers.
  2. E. R. Tufte. The Visual Display of Quantitative Information. Graphics Press, Cheshire, CT, USA, 198
  3. Jock Mackinlay. 1986. Automating the design of graphical presentations of relational information. ACM Trans. Graph. 5, 2 (April 1986), 110–141. doi:10.1145/22949.22950.
  4. Cleveland, W., & McGill, R. (1984). Graphical Perception: Theory, Experimentation, and Application to the Development of Graphical Methods. Journal of the American Statistical Association, 79(387), 531-554. doi:10.2307/2288400/
  5. David Braddock, Richard Hemp, Amie Lulinski, and Emily Tanis. 2017. The State of the States in Intellectual and Developmental Disabilities: 2017.
  6. 2020. Autism Data Visualization Tool. https://www.cdc.gov/ncbddd/autism/data/index.html .
  7. Jeffrey Heer, Frank Ham, Sheelagh Carpendale, Chris Weaver, and Petra Isenberg. 2008. Creation and Collaboration: Engaging New Audiences for Information Visualization. Information Visualization: Human-Centered Issues and Perspectives. Springer-Verlag, Berlin, Heidelberg, 92–133. doi: https://doi.org/10.1007/978-3-540-70956-5_5
  8. Keke Wu, Emma Petersen, Tahmina Ahmad, David Burlinson, Shea Tanis, and Danielle Albers Szafir. 2021. Understanding Data Accessibility for People with Intellectual and Developmental Disabilities. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI '21). Association for Computing Machinery, New York, NY, USA, Article 606, 1–16. doi: https://doi.org/10.1145/3411764.3445743
  9. J. Zacks and B. Tversky. Bars and lines: A study of graphic communication. Memory & Cognition, 27(6):1073–1079, Nov 1999. doi: 10.3758/BF03201236
  10. R. Kosara. The Impact of Distribution and Chart Type on Part-to-Whole Comparisons. J.Johansson, F. Sadlo, and G. E. Marai, eds., EuroVis 2019 - Short Papers. The Eurographics Association, 2019. doi:10.2312/evs.20191162
  11. M. A. Borkin, Z. Bylinskii, N. W. Kim, C. M. Bainbridge, C. S. Yeh, D. Borkin, H. Pfister, and A. Oliva. Beyond memorability: Visualization recognition and recall. IEEE TVCG,22(1):519–528, 2016.
  12. M. A. Borkin, A. A. Vo, Z. Bylinskii, P. Isola, S. Sunkavalli, A. Oliva, and H. Pfister. What makes a visualization memorable? IEEE TVCG, 19(12):2306–2315, 2013.
  13. J. Hullman, E. Adar, and P. Shah. Benefitting infovis with visual difficulties. IEEE Transactions on Visualization and Computer Graphics,17(12):2213–2222, Dec. 2011. doi:10.1109/TVCG.2011.175
  14. Burmeister, Oliver. (2010). Websites for Seniors: Cognitive Accessibility Websites for Seniors: Cognitive Accessibility. Australian Journal of Emerging Technologies and Society.
  15. Delinda Van Garderen. 2006. Spatial Visualization, Visual Imagery, and Mathe¬matical Problem Solving of Students With Varying Abilities. Journal of Learning Disabilities 39 (12 2006), 496–506. https://doi.org/10.1177/00222194060390060201
  16. R. Kosara and J. Mackinlay. Storytelling: The next step for visualization. Computer, (5):44–50, 2013.
  17. B. Lee, N. H. Riche, P. Isenberg and S. Carpendale, "More Than Telling a Story: Transforming Data into Visually Shared Stories," in IEEE Computer Graphics and Applications, vol. 35, no. 5, pp. 84-90, Sept.-Oct. 2015, doi: 10.1109/MCG.2015.99.
  18. 2015. Cognitive Accessibility User Research. https://www.w3.org/TR/coga-user¬research/#research-on-cognitive-function /
  19. Peck, Evan & Ayuso, Sofia & El-Etr, Omar. (2019). Data is Personal: Attitudes and Perceptions of Data Visualization in Rural Pennsylvania. CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1-12.10.1145/3290605.3300474.
  20. Roberts, Jonathan & Headleand, Chris & Perkins, David & Ritsos, Panagiotis. (2015). Personal Visualization for Learning.
  21. Katharina Rall, Margaret L. Satterthwaite, Anshul Vikram Pandey, John Emerson, Jeremy Boy, Oded Nov, Enrico Bertini, Data Visualization for Human Rights Advocacy, Journal of Human Rights Practice, Volume 8, Issue 2, July 2016, Pages 171–197, https://doi.org/10.1093/jhuman/huw011
  22. Zhang, D., Ding, Y., Stegall, J. and Mo, L. (2012), The Effect of Visual-Chunking-Representation Accommodation on Geometry Testing for Students with Math Disabilities. Learning Disabilities Research & Practice, 27: 167-177. https://doi.org/10.1111/j.1540-5826.2012.00364.x
  23. Steve Haroz, Robert Kosara, and Steven L. Franconeri. 2015. ISOTYPE Visualization: Working Memory, Performance, and Engagement with Pictographs. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). Association for Computing Machinery, New York, NY, USA, 1191–1200. doi: https://doi.org/10.1145/2702123.2702275
  24. Hamidi, Foad & Owuor, Patrick & Onyango, Deurence & Hynie, Michaela & Mcgrath, Susan & Baljko, Melanie. (2018). Participatory design of DIY digital assistive technology in Western Kenya. 1-11. 10.1145/3283458.3283478.
  25. González, H.S., Córdova, V.V., Cid, K.E., Azagra, M.J., & Álvarez-Aguado, I. (2020). Including intellectual disability in participatory design processes: Methodological adaptations and supports. Proceedings of the 16th Participatory Design Conference 2020 - Participation(s) Otherwise - Volume 1.
  26. Dear Data: http://www.dear-data.com/theproject

About the Authors

Keke Wu is a PhD student in Creative Technology & Design at University of Colorado Boulder working with Dr. Danielle Albers Szafir. Her research looks at Visualization & Cognitive Accessibility, particularly through the lens of multimedia design & storytelling. Keke believes in the power of visual cognition and communication, she hopes to capture the diversity and vibrancy of the disabled community and help people of all ability levels reach their full potential with technology.