Using functional assessment to improve information systems research, design and technology matching

Kathleen J. Price

Interactive Systems Research Center Department of Information Systems, UMBC Baltimore, MD 21250

E-mail: kprice1@umbc.edu

Abstract

Information systems researchers and developers require an appropriate means of assessing system user characteristics and capabilities. At this time, no such assessment measures exist. Current measurement tools are designed to assess activities of daily living, but do not transfer well to technology assessment. My dissertation work will focus on the development of a functional assessment (FA) that focuses on the capabilities that are critical for effective interactions with information technologies. This dissertation will focus on physical capabilities, using both questionnaire and computerized solutions to assess a user's relevant functional capabilities. Evaluation and comparison of assessment techniques with correlation to representative computer tasks will finalize the research.

Introduction and problem description

The scope of human computer interaction (HCI) includes a determination of the appropriateness of a given technology for a user, but this evaluation is difficult without an accurate evaluation of a user's functional information technology (IT) capabilities. Functional capabilities are often viewed as two extremes of having fully functioning capabilities vs. being disabled, but in reality all IT users vary in their level of functioning along a continuum between the two extremes over time (Newell, 1995). For example, a user who may easily operate IT when young can have vision difficulties later in life which require changes to the user interface for optimal application. Given this variation of IT functional capabilities, it is necessary to perform accurate and consistent assessment of user capabilities to match consumers to optimal IT solutions. Consistent, reproducible, methods of assessing an individual's functional capabilities will also prove useful in supporting research activities as it would allow researchers to more effectively compare results between studies and could allow inclusion criteria to be defined based on what the individuals can do as opposed to their underlying clinical diagnosis.

Current FA methods are often used to assess daily activities such as the ability to dress or feed yourself. Physical motions required to complete such activities do not equal actions used in IT tasks. The inability to accurately assess user capabilities with IT hinders the completion and evaluation of HCI research. Inappropriate FA techniques thwart efforts to match users, of all abilities, with appropriate computer technologies. Finally, the inability to assess user capabilities impedes efforts to evaluate effectiveness of user-technology matching.

Background and literature overview

HCI researchers have investigated appropriate methods to design accessible IT for people with physical disabilities. For example, researchers have examined user interactions with existing devices such as keyboards (Trewin & Pain, 1998; Trewin, 1996), the development of assistive solutions to support the use of those devices such as software filters which reduce certain errors (Trewin, 2002), the applicability of user models (Keates, Clarkson & Robinson, 2000), and techniques for user-technology matching (Scherer, 2002).

Temporary disabilities caused by context of use can instigate functional disabilities with regard to IT use. Such disabilities are classified as a situationally-induced impairment and disability (SIID). Health conditions and/or context of use issues can create an IT user impairment (Sears & Young, 2003). The proposed assessments will also address such a temporary SIID.

A medical model using clinical diagnoses is often used to categorize users in order to conduct HCI research for users with disabilities and to compare results from different research studies. However, this method is not ideal because the level of granularity does not match what is needed for IT research. For example, a diagnosis of tremors can be associated with varying types of tremors (action, resting, intention, essential, senile and familial) of varying severity and frequency of occurrence (Law, Sears, & Price, 2005). Therefore, a diagnosis of tremors does not provide sufficient information to determine the person's ability to interact with IT. To date, no appropriate method of functional IT capability assessment has been documented.

Goal of the research

The goal of this research is to develop an accurate, repeatable, and comparable method of assessing user functional capabilities with regard to the use of IT devices.

Motivation

The motivation for this research project is the lack of adequate techniques to assess IT specific functional motor skills. The accurate FA of motor capabilities necessary for IT use is important to enable categorization of user performance when evaluating the merit of the HCI solutions in question, but is also essential for determining the validity of research and permitting comparison between a variety of studies. In addition, this information can be used for follow-up assessment of IT suitability for individual users.

Research questions

Initially, current FA techniques will be assessed to determine the scope of what is presently available for use by HCI professionals and the applicability of current techniques to the dilemma of assessing IT functional skills. Follow-up research is based on the belief that no appropriate FA techniques are currently available to assess user capabilities with IT. This assumption will be confirmed via the literature review. The essential research questions for this project are the following: What is the best method for assessing user IT functional capabilities? Will a function-based assessment prove more effective than the diagnosis-based assessments that are often used in ongoing research?

During this project a questionnaire for assessing IT functional capabilities will be developed. Based on the initial research and questionnaire development, a computer-based assessment tool will be developed to test one or more aspects of IT functional assessment. This research will conclude by investigating the suggested FA solutions via a study of selected users comparing the questionnaire-based solution, a technology-based solution, and a traditional diagnosis-based approach. A correlation to actual computer use will also be completed.

Current status of research

Currently the literature review and an iterative methodology are in process to develop the FA questionnaire. Next steps include development of a computer-based FA solution and the design of a research study to perform comparison of questionnaire, computer, and diagnosis-based solutions with correlation to IT tasks.

Preliminary results accomplished

Preliminary accomplishments include the delineation of functional motor capabilities to be evaluated and the beginning stages of questionnaire item development. No user testing has occurred at this time.

Open issues and expected achievements

Open issues include the completion and validation of the FA questionnaire, development of a computer-based FA solution, and an empirical study comparing the use of the questionnaire, computer-based FA, and traditional clinical diagnoses with correlation to IT tasks.

Acknowledgements

I would like to thank my mentor, Dr. Andrew Sears for his guidance and encouragement during the genesis of my research career. This research is supported by the National Science Foundation (NSF) under Grant No. IIS-0511954. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author and do not necessarily reflect the views of NSF.

References

1. Keates, S., Clarkson J., & Robinson, P. (2000). Investigating the Applicability of User Models for Motion-Impaired Users. Proceedings of the fourth international ACM conference on Assistive technologies (pp. 129-136). New York: ACM Press.
2. Newell, A. F. (1995). Extra-ordinary human-computer interaction. In A.D.N. Edwards (Ed.), Extra-ordinary human-computer interaction: Interfaces for users with disabilities (pp. 3-18). New York: Cambridge University Press.
3. Law, C. M., Sears, A., & Price, K. J. (2005). Issues in the categorization of disabilities for user testing. Proceedings of HCII 2005 (CD Version). Mahwah, NJ: Lawrence Erlbaum Associates.
4. Trewin, S. (1996). A study of input device manipulation difficulties. Proceedings of the second annual ACM conference on Assistive technologies (pp. 15-22). New York: ACM Press.
5. Trewin, S. (2002). An invisible keyguard. Proceedings of the fifth international ACM conference on Assistive technologies (pp. 143-149). New York: ACM Press.
6. Trewin, S., and Pain, H. (1998). A Model of Keyboard Configuration Requirements. Proceedings of the third international ACM conference on Assistive technologies (pp. 173-181). New York: ACM Press.
7. Scherer, M. (2002). Matching consumers with appropriate assistive technologies. In DeRuyter & Olson (Eds.) Clinician's guide to assistive technology (pp. 3-13). Philadelphia: Mosby.
8. Sears, A., & Young, M. (2003). Physical disabilities and computing technologies: An analysis of impairments. In J.A. Jacko & A. Sears (Eds.), The human-computer interaction handbook (pp. 482-503). Mahwah, NJ: Lawrence Erlbaum Associates.