engleski

Realization of multimodal behaviour of embodied conversational agents in real-time

ECAs are computer-generated virtual characters who express human-like abilities in interaction with humans: they speak, make gestures, reason about ongoing conversation and adapt it to their conversational goal. The process of making ECAs is challenging because it is multidisciplinary and requires fundamental advances in speech recognition, natural language understanding and generation, dialogue management, cognitive modelling and reasoning, virtual human architectures and computer graphics and animation. Therefore, present research evidences how research laboratories cooperate in order to create effective and realistic ECAs. Besides, ECA community attempts to set specifications and encourage researchers to exchange their ideas and ECA system components. We contribute to the vast area of ECA research by devising RealActor, an open-source character animation and behaviour system with support for modelling multimodal communication using Behaviour Markup Language (BML). The system incorporates synching mechanisms which realize multimodal utterances in real-time using animation examples, procedural animations, and techniques of motion blending. We present a novel solution for aligning animations with synthesized speech and explain our Facial Action Coding system (FACS)-based animation model modelled using techniques of face tracking. Due to scarcity of research on multiuser-ECA interaction we propose new approaches to development of multiuser support in an ECA system, experiment with different equipment for human-machine interaction and study impressions of interacting with such a system.

Embodied Conversational Agents (ECAs); multimodal behaviours; realization of communicative utterances; multiuser interaction

nije evidentirano