Member of SIGACCESS since: 2004

Member of ACM since: 2004

Also member of: IGCSE (Computer Science Education)

I've been working in the areas of Accessibility Technology and Natural Language Processing for eight years, and I've focussed on the issue of translating English text into American Sign Language (ASL) animations for the past four years.

I enjoy working in areas of computer science that have a clear social impact and which require researchers to keep the users of their technology in mind during the design process. Having studied ASL for the past several years, I have had the opportunity to better understand the life experiences and culture of the deaf. I saw how ASL animation technology could have useful applications for deaf users with low English literacy. I have found ASL to be a challenging and intriguing language to study from a computational linguistic perspective, and I believe that computationally modelling the visual/spatial properties of the language will ultimately lead to new developments in Natural Language Processing technology (with applications cross-linguistically). I also feel that we have only begun to discover the ways in which computational linguistic technology can have useful assistive/accessibility applications - I find exploring the intersection of these two fields is an exciting area of research.

I am currently developing machine translation technology to automatically translate from English text into American Sign Language (ASL) animations. This technology is useful to the many deaf people in the U.S. and Canada who have low levels of written-English literacy. ASL is a natural language with a linguistic structure and word-order that is often different than English, and so many deaf people who are fluent in ASL may have difficulty reading the English text on the web, on closed-captioned television, or on other accessibility devices for the deaf. Technology that converts English text strings into on-screen animations of a virtual human character performing ASL could be extremely valuable to these users.

Because ASL takes advantage of the visual/spatial modality and is performed with various parts of the body in parallel over time, it presents many challenges from a computational linguistic perspective. Traditional linguistic technology designed for written-languages is not directly applicable to ASL, and so a major theme of my work has been finding new ways to design the components of a natural language processing system to enable the development of English-to-ASL machine translation software. Some of the new technologies developed for this project include: a multi-pathway machine translation architecture, a 3D visualization of the arrangement of objects under discussion during an ASL conversation, an artificial-intelligence planning-based animation generator, and a multi-channel representation of the temporal structure of the ASL animation performance.

I've developed a working prototype system that can translate an initial corpus of English sentences, and I'm conducting a scalability study of the design. An evaluation study began in June 2006, during which native ASL signers critique the animations produced by the system. As a baseline for comparison, the study participants will also see animations produced by recording the movements (with a motion-capture body suit and data-gloves) of a human signer performing ASL. By comparing the animations produced by the system to the animations recorded from the human signer, we can evaluate the output of the system and direct its future development.

I graduated from University of Delaware in 2001 with a B.S. and M.S. in Computer and Information Science. On a George Mitchell Scholarship, I studied in Ireland during 2001-2002, and I received an M.Sc. in Computer Science from the National University of Ireland, University College Dublin. I entered the Ph.D. program in Computer Science at the University of Pennsylvania in 2002, and I began my work on American Sign Language shortly thereafter. I received a M.S.E. in Computer Science in December 2004, and I expect to graduate with a Ph.D. in Computer Science in August 2006. In September, I will begin as an Assistant Professor of Computer Science at the City University of New York (CUNY), Queens College, where I am establishing a research lab for assistive technology and sign language computational linguistics.

The first accessibility technology project that I worked on was as an undergraduate researcher for Dr. Kathleen McCoy's ICICLE project at the University

I've participated in the ASSETS conference for the past two years. In 2004, I presented an overview of my Ph.D. dissertation in the Doctoral Consortium; as the winner of the "Best Doctoral Candidate" award, I was invited to present my work during the closing plenary of the conference. The advice and feedback that I received during the conference was extremely valuable while I was developing my dissertation proposal.

Last year, I presented a paper describing a new formalism for representing the timing relationships between parts of an animated character's body during an American Sign Language performance. This representation has enabled my project to encode coordination and non-coordination relationships between elements of the animation output of the system. This paper received the "Best Paper Award" for 2005.

I'm looking forward to serving as one of the publicity chairs for the ASSETS 2007 conference.