Abstract
In this paper, a new and powerful learning algorithm, which combines the idea of constructive algorithm, functional-link neural network and VC dimension, is introduced. Its structure consists of two stages, a mapping stage and a learning stage, corresponding to three layers, namely, input layer, hidden layer and output layer. The input training vectors are initially mapped to the feature vectors in the mapping stage by multiplying with a random matrix, followed by a pointwise nonlinear transformation. This is done starting with only one hidden node. In the learning stage, the feature vectors are fed into the least squares learning module to obtain the estimated output vectors. This is made possible by choosing the output layer linear. The optimum dimension of the feature space is determined by either testing against a separate validation set or using the idea of VC dimension. In this way, the number of hidden nodes is also learned automatically. The proposed structure implements structural risk minimization principle, and therefore, high generalization accuracy is guaranteed. It is also simple and extremely fast to compute, and is very suitable for many real-world applications.
Date of this Version
January 2002