engleski

On creating multimodal virtual humans - real time speech driven facial gesturing

In this paper we present a novel method for automatic speech driven facial gesturing for virtual humans capable of real time performance. Facial gestures included are various nods and head movements, blinks, eyebrow gestures and gaze. A mapping from speech to facial gestures is based on the prosodic information obtained from the speech signal. It is realized using a hybrid approach - HMMs, rules and global statistics. Further, we test the method using an application prototype - a system for speech driven facial gesturing suitable for virtual presenters.

facial gestures; vVisual prosody; multimodal interface; facial animation; speech processing; Human-Computer interaction

Special Issue on Multimodal Interaction and Multimodal Content Management