Multimodal Emotion Analysis Based on Acoustic and Linguistic Features of the Voice (CROSBI ID 704868)
Prilog sa skupa u zborniku | izvorni znanstveni rad | međunarodna recenzija
Podaci o odgovornosti
Koren, Leon ; Stipančić, Tomislav
engleski
Multimodal Emotion Analysis Based on Acoustic and Linguistic Features of the Voice
Artificial speech analysis can be used to detect non-verbal communication cues and reveal the current emotional state of the person. The inability of appropriate recognition of emotions can inevitably lessen the quality of social interaction. A better understanding of speech can be achieved by analyzing the additional characteristics, like tone, pitch, rate, intensity, meaning, etc. In a multimodal approach, sensing modalities can be used to alter the behavior of the system and provide adaptation to inconsistencies of the real world. A change detected by a single modality can generate a different system behavior at the global level. In this paper, we presented a method for emotion recognition based on acoustic and linguistic features of the speech. The presented voice modality is a part of the larger multi-modal computation architecture implemented on the real affective robot as a control mechanism for reasoning about the emotional state of the person in the interaction. While the audio is connected to the acoustic sub- modality, the linguistic sub-modality is related to text messages in which a dedicated NLP model is used. Both methods are based on neural networks trained on available open-source databases. These sub-modalities are then merged in a single voice modality through an algorithm for multimodal information fusion. The overall system is tested on recordings available through Internet services.
Emotion recognition ; Affective robotics ; Multimodal information fusion ; Voice analysis ; Speech recognition ; Learning ; Reasoning
nije evidentirano
nije evidentirano
nije evidentirano
nije evidentirano
nije evidentirano
nije evidentirano
Podaci o prilogu
301-311.
2021.
objavljeno
10.1007/978-3-030-77626-8_20
Podaci o matičnoj publikaciji
Social Computing and Social Media: Experience Design and Social Network Analysis. HCII 2021. Lecture Notes in Computer Science, vol 12774
Meiselwitz, Gabriele
Springer
978-3-030-77625-1
Podaci o skupu
13th International Conference Social Computing and Social Media (SCSM 2021)
predavanje
24.07.2021-29.07.2021
online
Povezanost rada
Informacijske i komunikacijske znanosti, Interdisciplinarne prirodne znanosti, Kognitivna znanost (prirodne, tehničke, biomedicina i zdravstvo, društvene i humanističke znanosti), Računarstvo, Strojarstvo