Naslov
Application of Deep Learning Transformer Model to Natural Language Processing

Autori
Belušić, Marko

Vrsta, podvrsta i kategorija rada
Ocjenski radovi, diplomski rad, preddiplomski

Fakultet
Fakultet organizacije i informatike

Mjesto
Varaždin

Datum
27.09

Godina
2021

Stranica
36

Mentor
Okreša Đurić, Bogdan ; Schatten, Markus

Ključne riječi
transformer ; chatbot ; deep learning ; transformer model ; transformer architecture ; AI ; NLP

Sažetak
This thesis presents why the transformer model of deep learning overshadowed its predecessors LSTMs and classic RNNs in many natural language processing by explaining in detail its inner workings. It covers the theoretical basis of the transformer model and how its components from the encoder to decoder work together to produce valuable results. The application of the transformer model is shown through a practical example named chatbot. The chatbot is implemented using Python and Tensorflow framework, and the base architecture is the transformer model. That example shows how attention is a powerful concept even with a small dataset.

Izvorni jezik
Engleski

Znanstvena područja
Informacijske i komunikacijske znanosti

POVEZANOST RADA