engleski

Parallel levenberg-marquardt-based neural network with variable decay rate

In this paper, parallel Levenberg-Marquardt-based feed-forward neural network with variable weight decay, implemented on the Graphics Proce-ssing Unit, is suggested. Two levels of parallelism are implemented in the algorithm. One level of parallelism is achieved across the data set, due to inherently parallel structure of the feed-forward neural networks. Another level of parallelism is achieved in Jacobian computation. To avoid third level of parallelism, i.e. parallelization of optimi-zation search steps, and to keep the algorithm simple, variable decay rate is used. Parameters of variable decay rate rule allow for compromise between oscillations and higher accuracy on one side and stable but slower convergence on the other side. To improve training speed and efficiency modification of random weight initializa-tion is included. Testing of a parallel algorithm is performed on two real domain benchmark problems. Results, given in a form of a table with obtained speedups, show the effectiveness of proposed algorithm implementation.

neural networks; regression; parallel levenberg-marquardt algorithm

nije evidentirano