Naslov
Machine Translation Evaluation: a Case study of Croatian-English and Russian-English MT Systems

Autori
Simeon, Ivana

Vrsta, podvrsta i kategorija rada
Sažeci sa skupova, sažetak, znanstveni

Skup
The Fundamentals of Verbal and Non-verbal Communication and the Biometrical Issue

Mjesto i datum
Vietri sul Mare, Italija, 02.09.2006. - 09.09.2006

Vrsta sudjelovanja
Predavanje

Vrsta recenzije
Međunarodna recenzija

Ključne riječi
machine translation; MT evaluation; evaluation criteria

Sažetak
From the earliest days of machine translation (MT), evaluation has been an inherent and significant part of efforts invested into machine translation research. In this paper, an overview of the history of MT evaluation is presented, with emphasis on one of the most comprehensive MT evaluation projects, undertaken in the 1960s, namely the Automatic Language Processing Advisory Committee Report. Furthermore, strategies and problems pertaining to MT evaluation are discussed, with emphasis on the distinction between subjective criteria, such as comprehensibility, and the objective, quantifiable criteria, such as error quantification and analysis. Within the practical part of the paper, the results of testing four MT systems – one for the language pair Croatian-English, and three for the language pair Russian-English – are shown. The systems were tested on three textual samples belonging to general, fictional and scientific genres. The analysis of the results included a comprehensibility poll which included five informants (native or proficient target language speakers) for each target language, as well as quantification of errors and error type assessment across genres and across individual MT systems. Finally, cumulative results are given for each MT system and for each language pair. As a conclusion, recent developments in the field of MT evaluation are presented, including automatic evaluation methods, such as IBM’ s measures BLEU and NIST.

Izvorni jezik
Engleski

Znanstvena područja
Filologija

POVEZANOST RADA