Naslov
Decentralized Trustless Gossip Training of Deep Neural Networks

Autori
Šajina, Robert ; Tanković, Nikola ; Etinger, Darko

Vrsta, podvrsta i kategorija rada
Radovi u zbornicima skupova, cjeloviti rad (in extenso), znanstveni

Izvornik
Proceedings of the 43rd International Convention on Information and Communication Technology, Electronics and Microelectronics / Skala, Karolj - Rijeka : Hrvatska udruga za informacijsku i komunikacijsku tehnologiju, elektroniku i mikroelektroniku - MIPRO, 2020, 1324-1328

Skup
43rd International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO 2020)

Mjesto i datum
Opatija, Hrvatska, 28.09.2020. - 02.10.2020

Vrsta sudjelovanja
Predavanje

Vrsta recenzije
Međunarodna recenzija

Ključne riječi
peer-to-peer network ; gossip protocol ; decentralized system ; neural-network ; machine learning

Sažetak
Novel machine learning techniques apply decentralized model training in order to mitigate data volume and privacy issues. Current approaches assume (a) node performance homogeneity, and (b) simultaneous training. These assumptions also imply that the predictive performance of the distributed models evolves uniformly. A different approach is required since a distributed decentralized network is heterogeneous and nonstationary: nodes can join or leave the network at any point in time (churn). We propose a novel protocol for exchanging the model knowledge between peers using a gossip algorithm combined with the stochastic gradient descent (SGD). Our method has the advantage of being fully asynchronous, decentralized, trustless, and independent of the network size and the churn ratio. We validated the proposed algorithm by running network simulations in various scenarios.

Izvorni jezik
Engleski

Znanstvena područja
Računarstvo, Informacijske i komunikacijske znanosti

POVEZANOST RADA