Multi-Modal Exploration

The University of Western Ontario
Canada
June 2006

PDF
We investigate the problem of providing accessible documents to visually impaired people through electronic mediation. In particular, we focus on the presentation of multi-modal documents through refreshing pin displays.

First, we examine the influence that refreshing pin display technology can have on the format of tactile picture. Inaccuracies in the reporting of finger positions from electronic tactile displays can result in errors in the audio presentation of multi-modal applications. We conduct an experiment to examine the accuracy of one such device. Given the results of this experiment, we present a collection of recommendations for the spacing of objects within a tactile scene.

Following this, we describe an algorithm for the detection of targets which will be encountered by a visually impaired user while exploring a two dimensional diagram on a refreshing tactile display. A user test examining the success of this algorithm during a targeted search task is described. We discuss the implications of this work on interface design for the visually impaired, including the planned inclusion of this algorithm in a multi-modal document browser.

Finally, we propose an architecture for multi-modal document presentation. This architecture, and a prototype application based on it, provide a framework for future inclusion of these results in the presentation of documents to visually impaired people.