Naslov
Chatbot evaluation metrics: review paper

Autori
Peras, Dijana

Vrsta, podvrsta i kategorija rada
Radovi u zbornicima skupova, cjeloviti rad (in extenso), ostalo

Izvornik
Economic and Social Development (Book of Proceedings) / Veselica, Rozana ; Dukić, Gordana ; Hammes, Khalid - Zagreb : VADEA, 2018, 89-97

Skup
36th International Scientific Conference on Economic and Social Development: Building Resilient Society

Mjesto i datum
Zagreb, Hrvatska, 14.12.2018. - 15.12.2018

Vrsta sudjelovanja
Predavanje

Vrsta recenzije
Međunarodna recenzija

Ključne riječi
chatbot ; chatbot assessment ; chatbot performance ; evaluation metrics

Sažetak
The past few years popularity of chatbots is constantly growing, and companies have been focused on developing them more then ever. Therefore, it is not surprising to see the news about various aspects of chatbot, from design and development to commercialization and marketing, are being daily published. Nevertheless, the topic of the chatbot evaluation is very often neglected. The metrics that should be used to evaluate the success of a chatbot are not systematized nor unified. One way to solve this problem is to align the metrics to the different perspectives of the chatbot evaluation: user experience perspective, information retrieval perspective, linguistic perspective, technology perspective and business perspective. In order to build the evaluation framework, the following categories of chatbot should be analyzed: usability, performance, affect, satisfaction, accuracy, accessibility, efficiency, quality, quantity, relation, manner, grammatical accuracy, humanity and business value. This paper provides a review on the evaluation metrics available for measuring success of efforts invested in chatbot, and proposes the chatbot evaluation framework based on five perspectives. The contribution of this paper is to help researchers to identify opportunities for the future research in evaluation of chatbot performance.

Izvorni jezik
Engleski

Znanstvena područja
Informacijske i komunikacijske znanosti

POVEZANOST RADA