This paper describes the FAME multi-modal demonstrator, which integrates multiple communication modes - vision, speech and object manipulation - by combining the physical and virtual worlds to provide support for multi-cultural or multi-lingual communication and problem solving. The major challenges are automatic perception of human actions and understanding of dialogs between people from different cultural or linguistic backgrounds. The system acts as an information butler, which demonstrates context awareness using computer vision, speech and dialog modeling. The integrated computer-enhanced human-to-human communication has been publicly demonstrated at the FORUM2004 in Barcelona and at IST2004 in The Hague. Specifically, the Interactive Space described features an Augmented Table for multi-cultural interaction, which allows several users at the same time to perform multi-modal, cross-lingual document retrieval of audio-visual documents previously recorded by an Intelligent Cameraman during a week-long seminar.
['Florian Metze', 'Petra Gieselmann', 'Hartwig Holzapfel', 'Tobias Kluge', 'Ivica Rogina', 'Alex Waibel', 'Matthias Wölfel', 'James L. Crowley', 'Patrick Reignier', 'Dominique Vaufreydaz', 'François Bérard', 'Bérangère Cohen', 'Joëlle Coutaz', 'Sylvie Rouillard', 'Victoria Arranz', 'Manuel Bertrán', 'Horacio Rodríguez']
The FAME interactive space