Pretražite po imenu i prezimenu autora, mentora, urednika, prevoditelja

Napredna pretraga

Pregled bibliografske jedinice broj: 1135086

Multimodal Emotion Analysis Based on Acoustic and Linguistic Features of the Voice


Koren, Leon; Stipančić, Tomislav
Multimodal Emotion Analysis Based on Acoustic and Linguistic Features of the Voice // Social Computing and Social Media: Experience Design and Social Network Analysis. HCII 2021. Lecture Notes in Computer Science, vol 12774 / Meiselwitz, Gabriele (ur.).
Virtual Event: Springer International Publishing, 2021. str. 301-311 doi:10.1007/978-3-030-77626-8_20 (predavanje, međunarodna recenzija, cjeloviti rad (in extenso), znanstveni)


CROSBI ID: 1135086 Za ispravke kontaktirajte CROSBI podršku putem web obrasca

Naslov
Multimodal Emotion Analysis Based on Acoustic and Linguistic Features of the Voice

Autori
Koren, Leon ; Stipančić, Tomislav

Vrsta, podvrsta i kategorija rada
Radovi u zbornicima skupova, cjeloviti rad (in extenso), znanstveni

Izvornik
Social Computing and Social Media: Experience Design and Social Network Analysis. HCII 2021. Lecture Notes in Computer Science, vol 12774 / Meiselwitz, Gabriele - : Springer International Publishing, 2021, 301-311

ISBN
978-3-030-77625-1

Skup
13th International Conference, SCSM 2021, Held as Part of the 23rd HCI International Conference, HCII 2021

Mjesto i datum
Virtual Event, 24-29.07.2021

Vrsta sudjelovanja
Predavanje

Vrsta recenzije
Međunarodna recenzija

Ključne riječi
Emotion recognition ; Affective robotics ; Multimodal information fusion ; Voice analysis ; Speech recognition ; Learning ; Reasoning

Sažetak
Artificial speech analysis can be used to detect non-verbal communication cues and reveal the current emotional state of the person. The inability of appropriate recognition of emotions can inevitably lessen the quality of social interaction. A better understanding of speech can be achieved by analyzing the additional characteristics, like tone, pitch, rate, intensity, meaning, etc. In a multimodal approach, sensing modalities can be used to alter the behavior of the system and provide adaptation to inconsistencies of the real world. A change detected by a single modality can generate a different system behavior at the global level. In this paper, we presented a method for emotion recognition based on acoustic and linguistic features of the speech. The presented voice modality is a part of the larger multi-modal computation architecture implemented on the real affective robot as a control mechanism for reasoning about the emotional state of the person in the interaction. While the audio is connected to the acoustic sub- modality, the linguistic sub-modality is related to text messages in which a dedicated NLP model is used. Both methods are based on neural networks trained on available open-source databases. These sub-modalities are then merged in a single voice modality through an algorithm for multimodal information fusion. The overall system is tested on recordings available through Internet services.

Izvorni jezik
Engleski

Znanstvena područja
Interdisciplinarne prirodne znanosti, Računarstvo, Strojarstvo, Informacijske i komunikacijske znanosti, Kognitivna znanost (prirodne, tehničke, biomedicina i zdravstvo, društvene i humanističke znanosti)



POVEZANOST RADA


Projekti:
HRZZ-UIP-2020-02-7184 - Afektivna multimodalna interakcija temeljena na konstruiranoj robotskoj spoznaji (AMICORC) (Stipančić, Tomislav, HRZZ - 2020-02) ( POIROT)

Ustanove:
Fakultet strojarstva i brodogradnje, Zagreb

Profili:

Avatar Url Tomislav Stipančić (autor)

Avatar Url Leon Koren (autor)

Poveznice na cjeloviti tekst rada:

doi link.springer.com

Citiraj ovu publikaciju:

Koren, Leon; Stipančić, Tomislav
Multimodal Emotion Analysis Based on Acoustic and Linguistic Features of the Voice // Social Computing and Social Media: Experience Design and Social Network Analysis. HCII 2021. Lecture Notes in Computer Science, vol 12774 / Meiselwitz, Gabriele (ur.).
Virtual Event: Springer International Publishing, 2021. str. 301-311 doi:10.1007/978-3-030-77626-8_20 (predavanje, međunarodna recenzija, cjeloviti rad (in extenso), znanstveni)
Koren, L. & Stipančić, T. (2021) Multimodal Emotion Analysis Based on Acoustic and Linguistic Features of the Voice. U: Meiselwitz, G. (ur.)Social Computing and Social Media: Experience Design and Social Network Analysis. HCII 2021. Lecture Notes in Computer Science, vol 12774 doi:10.1007/978-3-030-77626-8_20.
@article{article, author = {Koren, Leon and Stipan\v{c}i\'{c}, Tomislav}, editor = {Meiselwitz, G.}, year = {2021}, pages = {301-311}, DOI = {10.1007/978-3-030-77626-8\_20}, keywords = {Emotion recognition, Affective robotics, Multimodal information fusion, Voice analysis, Speech recognition, Learning, Reasoning}, doi = {10.1007/978-3-030-77626-8\_20}, isbn = {978-3-030-77625-1}, title = {Multimodal Emotion Analysis Based on Acoustic and Linguistic Features of the Voice}, keyword = {Emotion recognition, Affective robotics, Multimodal information fusion, Voice analysis, Speech recognition, Learning, Reasoning}, publisher = {Springer International Publishing}, publisherplace = {Virtual Event} }
@article{article, author = {Koren, Leon and Stipan\v{c}i\'{c}, Tomislav}, editor = {Meiselwitz, G.}, year = {2021}, pages = {301-311}, DOI = {10.1007/978-3-030-77626-8\_20}, keywords = {Emotion recognition, Affective robotics, Multimodal information fusion, Voice analysis, Speech recognition, Learning, Reasoning}, doi = {10.1007/978-3-030-77626-8\_20}, isbn = {978-3-030-77625-1}, title = {Multimodal Emotion Analysis Based on Acoustic and Linguistic Features of the Voice}, keyword = {Emotion recognition, Affective robotics, Multimodal information fusion, Voice analysis, Speech recognition, Learning, Reasoning}, publisher = {Springer International Publishing}, publisherplace = {Virtual Event} }

Časopis indeksira:


  • Current Contents Connect (CCC)
  • Web of Science Core Collection (WoSCC)
    • Science Citation Index Expanded (SCI-EXP)
    • SCI-EXP, SSCI i/ili A&HCI
    • Conference Proceedings Citation Index - Science (CPCI-S)
  • Scopus


Citati:





    Contrast
    Increase Font
    Decrease Font
    Dyslexic Font