Pretražite po imenu i prezimenu autora, mentora, urednika, prevoditelja

Napredna pretraga

Pregled bibliografske jedinice broj: 1066254

Clash between Segment-level MT Error Analysis and Selected Lexical Similarity Metrics


Brkic Bakaric, Marija; Tonkovic, Kristina; Nacinovic Prskalo, Lucia
Clash between Segment-level MT Error Analysis and Selected Lexical Similarity Metrics // International Journal of Advanced Computer Science and Applications, 11 (2020), 5; 35-42 doi:10.14569/ijacsa.2020.0110506 (međunarodna recenzija, članak, znanstveni)


CROSBI ID: 1066254 Za ispravke kontaktirajte CROSBI podršku putem web obrasca

Naslov
Clash between Segment-level MT Error Analysis and Selected Lexical Similarity Metrics

Autori
Brkic Bakaric, Marija ; Tonkovic, Kristina ; Nacinovic Prskalo, Lucia

Izvornik
International Journal of Advanced Computer Science and Applications (2158-107X) 11 (2020), 5; 35-42

Vrsta, podvrsta i kategorija rada
Radovi u časopisima, članak, znanstveni

Ključne riječi
Machine translation ; evaluation ; error analysis ; BLEU ; CHRF++ ; MQM

Sažetak
The aim of this paper is to evaluate the quality of popular machine translation engines on three texts of different genre in a scenario in which both source and target languages are morphologically rich. Translations are obtained from Google Translate and Microsoft Bing engines and German-Croatian is selected as the language pair. The analysis entails both human and automatic evaluation. The process of error analysis, which is time-consuming and often tiresome, is conducted in the userfriendly Windows 10 application TREAT. Prior to annotation, training is conducted in order to familiarize the annotator with MQM, which is used in the annotation task, and the interface of TREAT. The annotation guidelines elaborated with examples are provided. The evaluation is also conducted with automatic metrics BLEU and CHRF++ in order to assess their segmentlevel correlation with human annotations on three different levels–accuracy, mistranslation, and the total number of errors. Our findings indicate that neither the total number of errors, nor the most prominent error category and subcategory, show consistent and statistically significant segment-level correlation with the selected automatic metrics.

Izvorni jezik
Engleski

Znanstvena područja
Informacijske i komunikacijske znanosti



POVEZANOST RADA


Ustanove:
Sveučilište u Rijeci - Odjel za informatiku

Poveznice na cjeloviti tekst rada:

doi thesai.org

Citiraj ovu publikaciju:

Brkic Bakaric, Marija; Tonkovic, Kristina; Nacinovic Prskalo, Lucia
Clash between Segment-level MT Error Analysis and Selected Lexical Similarity Metrics // International Journal of Advanced Computer Science and Applications, 11 (2020), 5; 35-42 doi:10.14569/ijacsa.2020.0110506 (međunarodna recenzija, članak, znanstveni)
Brkic Bakaric, M., Tonkovic, K. & Nacinovic Prskalo, L. (2020) Clash between Segment-level MT Error Analysis and Selected Lexical Similarity Metrics. International Journal of Advanced Computer Science and Applications, 11 (5), 35-42 doi:10.14569/ijacsa.2020.0110506.
@article{article, year = {2020}, pages = {35-42}, DOI = {10.14569/ijacsa.2020.0110506}, keywords = {Machine translation, evaluation, error analysis, BLEU, CHRF++, MQM}, journal = {International Journal of Advanced Computer Science and Applications}, doi = {10.14569/ijacsa.2020.0110506}, volume = {11}, number = {5}, issn = {2158-107X}, title = {Clash between Segment-level MT Error Analysis and Selected Lexical Similarity Metrics}, keyword = {Machine translation, evaluation, error analysis, BLEU, CHRF++, MQM} }
@article{article, year = {2020}, pages = {35-42}, DOI = {10.14569/ijacsa.2020.0110506}, keywords = {Machine translation, evaluation, error analysis, BLEU, CHRF++, MQM}, journal = {International Journal of Advanced Computer Science and Applications}, doi = {10.14569/ijacsa.2020.0110506}, volume = {11}, number = {5}, issn = {2158-107X}, title = {Clash between Segment-level MT Error Analysis and Selected Lexical Similarity Metrics}, keyword = {Machine translation, evaluation, error analysis, BLEU, CHRF++, MQM} }

Časopis indeksira:


  • Web of Science Core Collection (WoSCC)
    • Emerging Sources Citation Index (ESCI)
  • Scopus


Citati:





    Contrast
    Increase Font
    Decrease Font
    Dyslexic Font