Search CALO Publications:

Author and Title
Paulo Barthelmess, Ed Kaiser, Xiao Huang, David Demirdjian. Distributed Pointing for Multimodal Collaboration over Sketched Diagrams. Proceedings of the Seventh International Conference on Multimodal Interfaces (ICMI '05), Trento, Italy, October 4-6, 2005, pgs. 10-17.

Abstract
A problem faced by groups that are not co-located but needto collaborate on a common task is the reduced access tothe rich multimodal communicative context that they wouldhave access to if they were collaborating face-to-face. Collaborationsupport tools aim to reduce the adverse effectsof this restricted access to the fluid intermixing of speech,
gesturing, writing and sketching by providing mechanisms to enhance the awareness of distributed participants of each others’ actions.

In this work we explore novel ways to leverage the capabilitiesof multimodal context-aware systems to bridge colocatedand distributed collaboration contexts. We describea system that allows participants at remote sites to collaborate in building a project schedule via sketching on multiple distributed whiteboards, and show how participants can be made aware of naturally occurring pointing gestures that reference diagram constituents as they are performed by remote participants.

The system explores the multimodal fusion of pen, speech and 3D gestures, coupled to the dynamic construction of a semantic representation of the interaction, anchored on the sketched diagram, to provide feedback that overcomes some of the intrinsic ambiguities of pointing gestures.

Download