TY - JOUR
T1 - Stability Analysis of the Modified Levenberg-Marquardt Algorithm for the Artificial Neural Network Training
AU - Rubio, Jose De Jesus
N1 - Publisher Copyright:
© 2012 IEEE.
PY - 2021/8
Y1 - 2021/8
N2 - The Levenberg-Marquardt and Newton are two algorithms that use the Hessian for the artificial neural network learning. In this article, we propose a modified Levenberg-Marquardt algorithm for the artificial neural network learning containing the training and testing stages. The modified Levenberg-Marquardt algorithm is based on the Levenberg-Marquardt and Newton algorithms but with the following two differences to assure the error stability and weights boundedness: 1) there is a singularity point in the learning rates of the Levenberg-Marquardt and Newton algorithms, while there is not a singularity point in the learning rate of the modified Levenberg-Marquardt algorithm and 2) the Levenberg-Marquardt and Newton algorithms have three different learning rates, while the modified Levenberg-Marquardt algorithm only has one learning rate. The error stability and weights boundedness of the modified Levenberg-Marquardt algorithm are assured based on the Lyapunov technique. We compare the artificial neural network learning with the modified Levenberg-Marquardt, Levenberg-Marquardt, Newton, and stable gradient algorithms for the learning of the electric and brain signals data set.
AB - The Levenberg-Marquardt and Newton are two algorithms that use the Hessian for the artificial neural network learning. In this article, we propose a modified Levenberg-Marquardt algorithm for the artificial neural network learning containing the training and testing stages. The modified Levenberg-Marquardt algorithm is based on the Levenberg-Marquardt and Newton algorithms but with the following two differences to assure the error stability and weights boundedness: 1) there is a singularity point in the learning rates of the Levenberg-Marquardt and Newton algorithms, while there is not a singularity point in the learning rate of the modified Levenberg-Marquardt algorithm and 2) the Levenberg-Marquardt and Newton algorithms have three different learning rates, while the modified Levenberg-Marquardt algorithm only has one learning rate. The error stability and weights boundedness of the modified Levenberg-Marquardt algorithm are assured based on the Lyapunov technique. We compare the artificial neural network learning with the modified Levenberg-Marquardt, Levenberg-Marquardt, Newton, and stable gradient algorithms for the learning of the electric and brain signals data set.
KW - Error stability
KW - Levenberg-Marquardt
KW - Newton
KW - weights boundedness
UR - http://www.scopus.com/inward/record.url?scp=85091271984&partnerID=8YFLogxK
U2 - 10.1109/TNNLS.2020.3015200
DO - 10.1109/TNNLS.2020.3015200
M3 - Artículo
C2 - 32809947
AN - SCOPUS:85091271984
SN - 2162-237X
VL - 32
SP - 3510
EP - 3524
JO - IEEE Transactions on Neural Networks and Learning Systems
JF - IEEE Transactions on Neural Networks and Learning Systems
IS - 8
M1 - 9170566
ER -