Naslov
Croatian Language N-Gram System

Autori
Šandor Dembitz, Bruno Blašković, Gordan Gledec

Vrsta, podvrsta i kategorija rada
Poglavlja u knjigama, znanstveni

Knjiga
Advances in Knowledge-Based and Intelligent Information and Engineering Systems

Urednik/ci
Graña, Manuel ; Toro, Carlos ; Posada, Jorge ; Howlett, Robert J. ; Jain, Lakhmi C.

Izdavač
IOS Press

Grad
Amsterdam

Godina
2012

Raspon stranica
696-705

ISBN
978-1-61499-104-5

Ključne riječi
Croatian, lexical n-gram, language modeling, Heaps’ law

Sažetak
Large-scale n-gram models are available for a small number of languages. So far, Croatian was not one of them. The research presented in this paper describes the development of n-gram database system suitable for large-scale language modeling in Croatian. The process of n-gram collection relies on Croatian academic online spellchecker Hascheck, which has been publicly available since 1993, and is today a popular language service, with average daily traffic exceeding million tokens. The approach demonstrated in this paper eliminated the need of n-gram data cleaning in the post-processing phase, which is a serious issue in other languages. The spellchecker dynamics allowed Heaps’ law modeling to be applied to Croatian n-grams, which enabled the prediction of n-gram count growth.

Izvorni jezik
Engleski

Znanstvena područja
Računarstvo

POVEZANOST RADA