Member of SIGACCESS since: Not sure

Member of ACM since: 1994

Also member of: SIGART

About 20.

It seems that some things are so difficult for people with disabilities, but they shouldn’t be. I would like to find ways of applying knowledge of artificial intelligence and natural language processing to make some of those things easier. I am inspired when I see people achieve incredible things under very difficult circumstances.

My projects are primarily involved with communication. I will highlight three projects which fall into two different areas. The first area is Augmentative and Alternative Communication (AAC). This field is concerned with developing systems and techniques for communicating for people whose disability precludes them from speaking in an understandable fashion. Generally such systems allow the user to select some language elements and a voice synthesizer is used to "speak" the desired selection. Often people who use AAC have physical impairments that make selecting language elements very slow; thus communication rate is a serious issue. Challenges exist in trying to speed communication rate while balancing cognitive load and enabling the most natural conversation. Two projects are highlighted in the area of AAC. The second area is in Computer Aided Language Learning systems. In this case, the concern is with teaching English as a second language to people who are deaf and signers of American Sign Language (ASL).

Fringe word prediction and topic modeling. In many AAC systems, developers have provided mechanisms for getting at core vocabulary - that fairly small set of words that appear most often in communication. Mechanisms include abbreviation expansion and encodings such as "minspeak" which enable most common words to be accessed in one to three keystrokes. While individual core vocabulary items do occur with high frequency, are words which individually occur significantly less frequently, but which together account for a large portion of communication. In this project we are concerned with access to this significantly larger "fringe" vocabulary consisting of potentially hundreds of thousands of words. We apply statistical natural language processing techniques in order to develop mechanisms for predicting these words based on what has already been typed. These methods have attained significant theoretical keystroke savings (60% for a predication window of 6 using a trigram-based language model). A focus of the current work is to improve prediction further by adding "topic modeling" where vocabulary items more on topic are highlighted over non-topic items in an automatic fashion based on the previous conversation. We investigate various methods for modeling and applying topics and find that they provide increased prediction power.

Using prestored text with unfamiliar partners. One way to significantly speed communication rate is to anticipate text needs and prestore full sentences to be called up during a conversation. If this is done, it is inevitable that there will be times when the message that is stored is not exactly what is needed for the current conversation. E.g., the prestored message may contain some irrelevant information, may contain too much or too little information for the purposes of the exchange, or may contain unnecessary repetition because it is actually composed of several messages strung together. This project (undertaken with Jan Bedrosian from Western Michigan University and Linda Hoag from Kansas State) is a research project that looks at the tradeoffs associated with either quickly delivering a message as stored, or taking the time to edit the message so that it better fits the conversation. W measure attitudes of unfamiliar partners in public situations in an information exchange situation (e.g., between a customer using AAC and a clerk at a store). The hypothesis is that the best choices will be different depending on the type of pragmatic mismatch that exists with the current discourse situation and the prestored message. The goal is to identify the relative benefits of various tradeoffs in order to develop future technology that best supports the findings.

The ICICLE Project (Interactive Computer Identification and Correction of Language Errors). The long term goal of ICICLE (a research project in its prototype phase) is a computer-based "grammar" tutor geared toward people who are deaf signers of American Sign Language (ASL) writing in English. Written English is a difficult language for people who are deaf to acquire which is not surprising if one considers the lack of usable input a person who is deaf has in acquiring the language. ASL is a language that is very different from English in terms of its grammatical structure. The goal of ICICLE is to act like an intelligent grammar checker where the errors that are identified are those that are typical of the deaf population (a very different set of errors than would be expected from the native English speaking population). What we want is a system that will point out errors and offer tutoring on those aspects most likely to be helpful to the writer. The system takes research in second language acquisition into account, and will adapt its interactions as the student becomes more proficient in English. Most of the work to date has concentrated on user modeling aspects of the system - modeling a student’s grammar of English and how that grammar changes over time so as to effectively point out errors.

My degrees (BS University of Delaware, MS and PhD University of Pennsylvania) are all in computer science focused on the subfield of artificial intelligence called natural language processing. My studies have always concentrated on an interdisciplinary perspective with heavy doses of psychology and linguistics throughout my academic career.

I have been a member of the ASSETS Program Committee (2006, 2005, 2004) and have presented papers at the ASSETS Conference in 2000 and 1994.

I think a journal in this area is long overdue.