Naslov
Sentence Retrieval using Stemming and Lemmatization with Different Length of the Queries

Autori
Boban, Ivan ; Doko, Alen ; Gotovac, Sven

Izvornik
Advances in Science, Technology and Engineering Systems Journal (2415-6698) 5 (2020), 3; 349-354

Vrsta, podvrsta i kategorija rada
Radovi u časopisima, članak, znanstveni

Ključne riječi
Sentence retrieval ; TF-ISF ; Data pre-processing ; Stemming ; Lemmatization

Sažetak
In this paper we focus on Sentence retrieval which is similar to Document retrieval but with a smaller unit of retrieval. Using data pre- processing in document retrieval is generally considered useful. When it comes to sentence retrieval the situation is not that clear. In this paper we use TF-ISF (term frequency – inverse sentence frequency) method for sentence retrieval. As pre-processing steps, we use stop word removal and language modeling techniques: stemming and lemmatization. We also experiment with different query lengths. The results show that data pre-processing with stemming and lemmatization is useful with sentences retrieval as it is with document retrieval. Lemmatization produces better results with longer queries, while stemming shows worse results with longer queries. For the experiment we used data of the Text Retrieval Conference (TREC) novelty tracks.

Izvorni jezik
Engleski

Znanstvena područja
Računarstvo

POVEZANOST RADA