Naslov
Hybrid approach to real-time speech driven facial gesturing of virtual characters

Autori
Zorić, Goranka

Vrsta, podvrsta i kategorija rada
Ocjenski radovi, doktorska disertacija

Fakultet
Fakultet elektrotehnike i računarstva

Mjesto
Zagreb

Datum
09.07

Godina
2010

Stranica
85

Mentor
Pandžić, Igor

Ključne riječi
facial gestures; visual prosody; virtual characters

Sažetak
This thesis investigates automatic facial gesturing of virtual characters based only on a speech input. Facial gestures that has been included are various nods and head movements, eye blinks, eyebrow gestures and gaze, i.e.facial and head movements used in nonverbal communication that are connected to the syntactic and prosodic structure of the underlying speech rather than to semantic or emotions. The main issue in speech driven facial gesturing is an audio to visual mapping. In this thesis, a hybrid method for the real-time correlation of the speech signal with facial gestures has been proposed. An idea behind the proposed method is to include different issues that are considered important for visual prosody through three different modules. In a data-driven module facial gestures are correlated with prominent parts of the speech. A rule-based module consists of a set of rules, which among others include rules for punctuation and prolonged pauses. Additionally, in a statistics module, results of first two modules are fine-tuned and further shaped. In order to check how believable virtual characters are if animated using facial gestures obtained by the proposed method, a system for speech driven facial gesturing has been implemented, and used for a perceptual evaluation. Testing subjects generally considered facial gestures consistent and time aligned with the underlying speech, and a virtual human applying them is perceived as a natural. Moreover, the hypothesis set has been proved: adding facial gestures driven by the speech signal to animation of virtual characters using the proposed hybrid approach in real-time results with more coherent and appropriate facial movements then random or no movements. Further, the system for speech driven facial gesturing of virtual characters has been put in a context of virtual news presenters used in networked environments, since that is where the future research of this system streams next.

Izvorni jezik
Engleski

Znanstvena područja
Računarstvo

POVEZANOST RADA