Decentralized Trustless Gossip Training of Deep Neural Networks (CROSBI ID 695697)
Prilog sa skupa u zborniku | izvorni znanstveni rad | međunarodna recenzija
Podaci o odgovornosti
Šajina, Robert ; Tanković, Nikola ; Etinger, Darko
engleski
Decentralized Trustless Gossip Training of Deep Neural Networks
Novel machine learning techniques apply decentralized model training in order to mitigate data volume and privacy issues. Current approaches assume (a) node performance homogeneity, and (b) simultaneous training. These assumptions also imply that the predictive performance of the distributed models evolves uniformly. A different approach is required since a distributed decentralized network is heterogeneous and nonstationary: nodes can join or leave the network at any point in time (churn). We propose a novel protocol for exchanging the model knowledge between peers using a gossip algorithm combined with the stochastic gradient descent (SGD). Our method has the advantage of being fully asynchronous, decentralized, trustless, and independent of the network size and the churn ratio. We validated the proposed algorithm by running network simulations in various scenarios.
peer-to-peer network ; gossip protocol ; decentralized system ; neural-network ; machine learning
nije evidentirano
nije evidentirano
nije evidentirano
nije evidentirano
nije evidentirano
nije evidentirano
Podaci o prilogu
1324-1328.
2020.
objavljeno
10.23919/MIPRO48935.2020.9245248
Podaci o matičnoj publikaciji
Proceedings of the 43rd International Convention on Information and Communication Technology, Electronics and Microelectronics
Skala, Karolj
Rijeka: Hrvatska udruga za informacijsku i komunikacijsku tehnologiju, elektroniku i mikroelektroniku - MIPRO
1847-3938
1847-3946
Podaci o skupu
MIPRO 2020
predavanje
28.09.2020-02.10.2020
Opatija, Hrvatska