Labeling of Gestures in SmartKom − Concept of the Coding System

Authors: Silke Steininger

The SmartKom project is concerned with the development of an intelligent computer−user interface that allows a user to communicate almost naturally with an adaptive and self−explanatory dialogue system. Among other things the system will be able to analyze the gestural input of the user. To train a gesture analyzer, data is required, preferably realistic data. One of the tasks of our institute in the project is the collection and annotation of such data. Since the machine does not yet exist the data collection is done with help of so called Wizard of Oz−experiments: The system is simulated by humans (the "wizards") and the subjects are made believe that they interact with an existing machine. We record the subjects (video and audio) as they solve short tasks. The recordings are labeled off−line with respect to the gestures that the subjects used.