Beth Mynatt
Georgia Tech
Transforming Graphical User Interfaces into Auditory User Interfaces

Computers in non-desktop environments have become ubiquitous over the past five years. But now we are faced with the chore of learning three applications just to read electronic mail at our desk, on the phone, and with a pager. One vision for the future is that a set of applications would follow the user from the desk, to the car, to the warehouse, and to the board room. The application interfaces would be transformed to suit each new environment while remaining familiar to the user. One piece of this vision is transforming graphical interfaces into auditory interfaces.

In this talk, I will describe a methodology for automatically transforming graphical interfaces into auditory interfaces. This methodology addresses analyzing and modeling the existing graphical interface, as well as designing auditory interfaces to represent the interface model. In this transformation, the salient characteristics of the graphical interface are conveyed using combinations of speech and nonspeech auditory output. The underlying model of the graphical interface, an annotated tree structure, forms the basis for navigating the auditory interface. I have investigated these ideas by designing an interface for blind users accessing graphical applications. With my system, Mercator, blind users can work with auditory versions of the graphical applications used by their sighted colleagues.

Elizabeth Mynatt is a Research Scientist in the Graphics, Visualization and Usability Center at Georgia Tech. She directs the Multimedia Computing Group which focuses on research in auditory and collaborative interfaces. Elizabeth is the designer of the Mercator interface which transforms graphical interfaces into auditory interfaces that are accessible for blind computer users. Her main interest is in integrating computers into the physical environment. To that end, she will be continuing to investigate transforming user interfaces to meet user and environmental constraints, as well as designing peripheral interfaces for collaborative systems.