Naslov
Real-time language independent lip synchronization method using a genetic algorithm

Autori
Zorić, Goranka ; Pandžić, Igor

Izvornik
Signal Processing (0165-1684) 86 (2006), 12; 3644-3656

Vrsta, podvrsta i kategorija rada
Radovi u časopisima, članak, znanstveni

Ključne riječi
lip synchronization; lip sync; facial animation; MPEG-4 FBA; human-computer interaction; virtual characters; speech processing; neural networks; genetic algorithms

Sažetak
Lip synchronization is a method for the determination of the mouth and tongue motion during a speech. It is widely used in multimedia productions, and real time implementation is opening application possibilities in multimodal interfaces. We present an implementation of real time, language independent lip synchronization based on the classification of the speech signal, represented by MFCC vectors, into visemes using neural networks. Our implementation improves real time lip synchronization by using a genetic algorithm for obtaining a near optimal neural network topology. The automatic neural network configuration with genetic algorithms eliminates the need for tedious manual neural network design by trial and error and considerably improves the viseme classification results. Moreover, by the direct usage of visemes as the basic unit of the classification, computation overhead is reduced, since only visemes are used for the animation of the face. The results are obtained in comprehensive validation of the system using three different evaluation methods, two objective and one subjective. The obtained results indicate very good lip synchronization quality in real time conditions and for different languages, making the method suitable for a wide range of applications.

Izvorni jezik
Engleski

Znanstvena područja
Elektrotehnika, Računarstvo, Informacijske i komunikacijske znanosti

Napomena
Special issue of Signal Processing on Multimodal Human-Computer Interfaces

POVEZANOST RADA