Naslov
Visualizations of the training process of neural networks

Autori
Babić, Karlo ; Meštrović, Ana

Vrsta, podvrsta i kategorija rada
Radovi u zbornicima skupova, cjeloviti rad (in extenso), znanstveni

Izvornik
2019 42nd International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO) / Biljanović, Petar - Opatija : Institute of Electrical and Electronics Engineers (IEEE), 2019, 1619-1623

Skup
42nd International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO 2019)

Mjesto i datum
Opatija, Hrvatska, 20.05.2019. - 24.05.2019

Vrsta sudjelovanja
Predavanje

Vrsta recenzije
Međunarodna recenzija

Ključne riječi
data visualisation ; learning (artificial intelligence) ; logic gates ; natural language processing ; neural nets ; optimisation ; statistical analysis ; text analysis

Sažetak
In this paper we present an approach to the visualizations of the neural networks training process. The main goal of such visualizations is to better understand the training process of neural networks, and to help with the creation and optimization of new neural networks. We implemented a tool which can be used to create a large number of visualizations by using different combinations of created transformations (for example: diff, plot2, angle-path, vec-dist-dev). To be able to visualize the learning process, we had to record all the weights from every n th iteration. For experiments we used three neural networks: a neural network for digit recognition (DR), a neural network for word2vec representation of words (WV), and a neural network which learned the simple logical gate XOR, (XR). For the purpose of further comparisons, we also constructed one example of a damaged network (DR', WV', XR') for each of the three original networks (DR, WV, XR). We constructed over 30 visualizations and then chose 6 visualizations which could be easily interpreted and present them in this paper. The results indicate that there may exist some regularities in the visualizations of original networks and that damaged network visualizations differ from the original networks.

Izvorni jezik
Engleski

Znanstvena područja
Računarstvo, Informacijske i komunikacijske znanosti

POVEZANOST RADA