Accessibility and Acceptability of Voice Assistants for People with Acquired Brain Injury

Aris Malapaschas, University of Glasgow, aris.malapaschas@glasgow.ac.uk

Abstract

Acquired Brain Injury can cause cognitive impairments, such as problems with memory and scheduling, and executive dysfunction. Assistive Technologies (AT), usually in the form of prompting and reminding systems, can improve remembering and task completion in people with ABI. Studies, however, show that the use of AT in this context is rather low, and that there are several factors that impact their uptake and effectiveness, indicating there is room for better design. Voice Assistants (VA), i.e. smart speakers with integrated virtual assistants, offer a versatile set of tools to provide support to cognitively impaired users, through a more accessible and pervasive interaction. Research however shows that there are several obstacles that can prevent the uptake and effective use of VAs by people with ABI.

This research aims to explore how the conversational interfaces of VAs can be designed to become more accessible and acceptable by people with ABI. An online survey (n=114) and several focus groups (n=29) were conducted to gather requirements for the design of such a system in a user-centred way. Additionally, the key design considerations of a conversational UI for calendar management were investigated in a user study with students (n=16).

This research aims to explore how the conversational interfaces of VAs can be designed to become more accessible and acceptable by people with ABI. An online survey (n=114) and several focus groups (n=29) were conducted to gather requirements for the design of such a system in a user-centred way. Additionally, the key design considerations of a conversational UI for calendar management were investigated in a user study with students (n=16).

This paper outlines the background and motivation behind the research, gives a brief overview of the planned course of action, and presents the results up to this point. The envisioned contributions of this research to the field of HCI are also addressed.

1. Introduction

Acquired Brain Injury (ABI) refers to any trauma caused to the brain after birth. It can be a result of a head injury, a stroke, a brain-tumour, or a brain disease (e.g. encephalitis). The effects of a brain injury can vary from one case to another, but they typically include physical impairments (e.g. reduced mobility and vision problems) and loss of cognitive skills such as memory, scheduling, organisation and decision making. These effects reduce the independence of the ABI survivor, who may have to rely on regular support from others to perform everyday activities. This, in turn, can result in withdrawal, and psychological stress both for the person with the ABI and the person providing the support [3].

After their injury, people with ABI often attend a neuropsychological rehabilitation program, where they work together with health professionals and others to reduce the impact of their deficits on everyday life [12]. Rehabilitation is usually approached in a holistic way [2]. Cognitive Rehabilitation (CR) focuses specifically on cognitive functions like memory, attention, problem solving and communication. Additionally, people with ABI use external aids to support their cognitive performance. These can be paper-based tools (e.g. calendars and to-do lists) [7], or Assistive Technologies (AT), i.e. technology-based tools usually in the form of prompting systems [11]. Although the latter can improve the frequency of remembering and completing everyday activities more effectively than their non-technological equivalents [9], studies show that the uptake and use of AT by people with ABI is rather low, and can be prevented by several factors [1, 8]. Among the barriers that limit the use of AT are the reduced accessibility to mainstream devices (i.e. smartphones and computers) due to physical impairments, and the inefficient use of these devices due to reduced cognitive function.

Virtual assistants, i.e. autonomous software agents that can perform tasks or services for their users, have facilitated rehabilitation in different contexts [6, 10]. Voice-operated virtual assistants -or Voice Assistants (VAs)- like Apple's Siri, Amazon's Alexa, and the Google Assistant, offer a diverse and increasingly ubiquitous ecosystem for the development of AT for people with ABI, with several potential benefits: Hands-free and non-visual interaction can increase accessibility, and integration with IoT devices and third-party services makes them suitable for controlling smart-home environments, which can be helpful for users with disabilities [5]. Despite being widespread however, VAs are not used very often, and interaction with them can be rather problematic [4]. Moreover, speech based interfaces can affect memory load and cognitive performance [13].

In order to employ the advantages of VAs and develop AT that can be beneficial for people with ABI (and users with cognitive impairments in general) it is essential to investigate the interaction design of their conversational interfaces in a user-centred way, and find ways to bypass the barriers that might prevent their efficient use. The purpose of this research is to explore how the interaction design of VAs can be improved so that they can become more accessible and acceptable by people with ABI.

2. Proposed Solution

The research process will consist of three phases. The first phase is the gathering of the requirements. The objectives of this phase are: to examine the common difficulties in everyday life caused by the cognitive effects of a brain injury, and the use of different technological aids by people with ABI, in order to determine the appropriate contexts and conditions for the use of AT. This will be accomplished through the studying of the relevant literature, and a set of qualitative and exploratory studies (i.e. survey and focus groups).

The objective of the third phase of the research will be the development and evaluation of a prototype system which can dynamically adjust the conversational interface of a VA, depending on its user's performance. The results from phase 2 will be used to determine the different interaction characteristics that will be used, and in which conditions. The system will be evaluated through a set of case studies, where the user's success rate in competing the required tasks through the VA will be monitored.

3. Status of Research

The first stage of the research included:

The results of the above studies were used to create a set of attributes for a prompting and reminding AT that improves the everyday functioning of people with ABI. In particular, such a prompting system should aim to Support memory and initiation, by providing timely reminders to prompt action, help users concentrate on the task at hand until its completion, provide a goal-oriented sense of achievement and self-monitoring to increase motivation, and remind users of their behaviour issues and how to control them. In order to identify the key considerations of the interaction design of virtual assistants for people with ABI, an exploratory study with healthy students was conducted. In this study, the participants were asked to perform calendar management tasks on a smartphone and on a Google Home device, and reflect on their experience. Different ways to approach the interaction were also examined. Currently a similar study with people with ABI is being conducted, during which the participants use VAs to perform different tasks and their experience is examined.

Contributions

The envisioned contributions of this research to the fields of HCI, Accessibility and Assistive Technology are the following:

Moreover, the research will result in the implementation of a prototype Assistive Technology that can be used as a useful tool in the context of neuropsychological rehabilitation, and to increase the cognitive accessibility of VAs.

Motivation

The ASSETS 2018 Doctoral Consortium will be an excellent opportunity for the author to present his research ideas and plans to researchers with experience in the field of Assistive Technologies and Accessibility. This will enable him to improve his presentation skills, and receive valuable feedback that will help him head in the right direction. Accepting constructive critique from experts in the field and from other students, with similar interests but different backgrounds and perspective, will allow him to recognise potential weaknesses in his methods and improve them. Hearing about other students' projects and the methods they use will help the researcher gain ideas about ways to enrich and solidify his plans. Attendance in the conference will also allow the author to engage in valuable discussions with fellow researchers who have similar aspirations, receive their feedback and try to aid them in a similar way.

Acknowledgments

I would like to thank my supervisor Prof. Stephen Brewster for all the help and support, the DC chairs of ASSETS 2018 for giving me the opportunity to take part in this Doctoral Consortium, and all the DC panel members for giving me invaluable feedback and advice.

References

  1. Baldwin, V.N. et al. 2011. Factors influencing the uptake of memory compensations: A qualitative analysis. Neuropsychological Rehabilitation. 21, 4 (2011), 484–501. DOI:https://doi.org/10.1080/09602011.2011.582378.
  2. Ben-Yishay, Y. and Daniels-Zide, E. 2000. Examined lives: Outcomes after holistic rehabilitation. Rehabilitation Psychology (2000).
  3. Brodaty, H. and Donkin, M. 2009. Family caregivers of people with dementia. Dialogues in Clinical Neuroscience. 11, 2 (2009), 217–228.
  4. Cowan, B.R. et al. 2017. “What can i help you with?” Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services - MobileHCI ’17. (2017), 1–12. DOI:https://doi.org/10.1145/3098279.3098539.
  5. Dewsbury, G. and Linskell, J. 2011. Smart home technology for safety and functional independence: The UK experience. NeuroRehabilitation. (2011). DOI:https://doi.org/10.3233/NRE-2011-0653.
  6. Fitzpatrick, K.K. et al. 2017. Delivering Cognitive Behavior Therapy to Young Adults With Symptoms of Depression and Anxiety Using a Fully Automated Conversational Agent (Woebot): A Randomized Controlled Trial. JMIR Mental Health. 4, 2 (2017), e19. DOI:https://doi.org/10.2196/mental.7785.
  7. Fleming, J.M. et al. 2005. Prospective memory rehabilitation for adults with traumatic brain injury: A compensatory training programme. Brain Injury. 19, 1 (2005), 1–10. DOI:https://doi.org/10.1080/02699050410001720059.
  8. Jamieson, M. et al. 2017. Technological memory aid use by people with acquired brain injury. Neuropsychological Rehabilitation. 27, 6 (2017), 919–936. DOI:https://doi.org/10.1080/09602011.2015.1103760.
  9. Jamieson, M. et al. 2014. The efficacy of cognitive prosthetic technology for people with memory impairments: A systematic review and meta-analysis. Neuropsychological Rehabilitation. 24, 3–4 (2014), 419–444. DOI:https://doi.org/10.1080/09602011.2013.825632.
  10. Lucas, G.M. et al. 2017. Reporting Mental Health Symptoms: Breaking Down Barriers to Care with Virtual Human Interviewers. Frontiers in Robotics and AI. 4, October (2017), 1–9. DOI:https://doi.org/10.3389/frobt.2017.00051.
  11. Svoboda, E. et al. 2015. Long-term maintenance of smartphone and PDA use in individuals with moderate to severe memory impairment. Neuropsychological Rehabilitation. 25, 3 (2015), 353–373. DOI:https://doi.org/10.1080/09602011.2014.927368.
  12. Wilson, B.A. et al. 2009. Neuropsychological rehabilitation: Theory, models, therapy and outcome.
  13. Wolters, M. et al. 2009. Reducing working memory load in spoken dialogue systems. Interacting with Computers. 21, 4 (2009), 276–287. DOI:https://doi.org/10.1016/j.intcom.2009.05.009.

About the Author

I am a 3rd year PhD student at the Computing Science department at the University of Glasgow. My focus is on Assistive Technologies for people with Cognitive Impairments. I am generally interested in the design and development of new technologies and interfaces to aid people with different disabilities.