A method to determine the required number of neural-network training repetitions

M S Iyer; R R Rhinehart

doi:10.1109/72.750573

A method to determine the required number of neural-network training repetitions

IEEE Trans Neural Netw. 1999;10(2):427-32. doi: 10.1109/72.750573.

Authors

M S Iyer¹, R R Rhinehart

Affiliation

¹ Department of Chemical Engineering, Texas Tech University, Lubbock, TX 79409-3121, USA.

PMID: 18252540
DOI: 10.1109/72.750573

Abstract

Conventional neural-network training algorithms often get stuck in local minima. To find the global optimum, training is conventionally repeated with ten, or so, random starting values for the weights. Here we develop an analytical procedure to determine how many times a neural network needs to be trained, with random starting weights, to ensure that the best of those is within a desirable lower percentile of all possible trainings, with a certain level of confidence. The theoretical developments are validated by experimental results. While applied to neural-network training, the method is generally applicable to nonlinear optimization.