Title: IMPROVING THE ACCURACY OF GRADIENT DESCENT BACK PROPAGATION ALGORITHM (GDAM) ON CLASSIFICATION PROBLEMS

Issue Number: Vol. 1, No. 4
Year of Publication: Dec - 2011
Page Numbers: 838-847
Authors: M. Z. Rehman, N. M. Nawi
Journal Name: International Journal of New Computer Architectures and their Applications (IJNCAA)
- Hong Kong

Abstract:


The traditional Back-propagation Neural Network (BPNN) Algorithm is widely used in solving many real time problems in world. But BPNN possesses a problem of slow convergence and convergence to local minima. Previously, several modifications are suggested to improve the convergence rate of Gradient Descent Back-propagation algorithm such as careful selection of initial weights and biases, learning rate, momentum, network topology, activation function and ‘gain’ value in the activation function. This research proposed an algorithm for improving the current working performance of Back-propagation algorithm by adaptively changing the momentum value and at the same time keeping the ‘gain’ parameter fixed for all nodes in the neural network. The performance of the proposed method known as ‘Gradient Descent Method with Adaptive Momentum (GDAM)’ is compared with the performances of ‘Gradient Descent Method with Adaptive Gain (GDM-AG)’ and ‘Gradient Descent with Simple Momentum (GDM)’. The efficiency of the proposed method is demonstrated by simulations on five classification problems. Results show that GDAM can be used as an alternative approach for BPNN because it demonstrate better accuracy ratio on the chosen classification problems.