Parameter Conjugate Gradient with Secant Equation Based Elman Neural Network and its Convergence Analysis

Qinwei Fan, Zhiwen Zhang, Xiaodi Huang

Research output: Contribution to journalArticlepeer-review

9 Citations (Scopus)

Abstract

Abstract Elman neural network (ENN) is one of the local recursive networks with a feedback mechanism. The parameter conjugate gradient method is a promising alternative to the gradient descent method, due to its faster convergence speed that results from searching for the conjugate descent direction with an adaptive step size (obtained by Wolfe conditions). However, there are still some challenges such as how to avoid the sawtooth phenomenon in gradient algorithms to improve the learning accuracy of the second-order curvature of an objective function. As such, this paper presents a novel parametric conjugate gradient method that is based on the secant equation for training ENN in an effective way. Strict proof of the theoretical convergence of the proposed algorithm is provided in detail. In particular, the weak convergence and strong convergence of the algorithm, as well as the monotonicity of the error function are proved. Except for the theoretical analysis, the three numerical experiments have been conducted by applying the algorithm to three problems of classification, regression, and function approximation on nine real-world datasets. The experimental results have demonstrated the feasibility of the proposed algorithm and the correctness of this theoretical analysis.
Original languageEnglish
JournalAdvanced Theory and Simulations
Volumen/a
Issue numbern/a
DOIs
Publication statusPublished - Jul 2022

Fingerprint

Dive into the research topics of 'Parameter Conjugate Gradient with Secant Equation Based Elman Neural Network and its Convergence Analysis'. Together they form a unique fingerprint.

Cite this