Adaptive self-organizing neural networks for matrix eigen-decomposition problems and their application to feature extraction

Chanchal Chatterjee, Purdue University

Abstract

We describe artificial neural networks and self-organizing learning algorithms to adaptively solve a set of matrix algebra problems. We discuss algorithms to compute the following matrix functions: (1) the normalized mean of a data sequence, (2) the inverse of the square root of the positive definite correlation matrix of a data sequence, and (3) several algorithms to compute the generalized eigenvectors of two correlation matrices of two data sequences. Although several applications are mentioned for these algorithms, we have used them primarily to obtain adaptive estimates of class-separability features. The feature extraction networks discussed in this study are: (1) a network for normalized correlation features, (2) networks for unimodal and multi-cluster Gaussian data in the multi-class case, (3) a network for multivariate linear discriminant analysis (LDA) in the multi-class case, (4) a network for Bhattacharyya distance measure for the two-class case, and (5) a network for multivariate LDA derived from a hetero-associative supervised network. For each algorithm, the convergence with probability one is proven by using stochastic approximation theory, and a single layer linear network architecture for the algorithm is described. In some cases, such as LDA, combinations of these algorithms are used to extract the features. In these cases, our methods allow for simultaneous training of the multi-layer networks. Convergence of the networks under simultaneous training is also proven. A key property of our training procedures is that they are adaptive in nature and hence, they are well-suited for online applications and can be easily implemented by VLSI technologies. Every network considers a flow or sequence of inputs for training, thereby eliminating the need for a pooled data for training. Numerical studies on the performance of the networks for multi-class random data are presented.

Degree

Ph.D.

Advisors

Roychowdhury, Purdue University.

Subject Area

Electrical engineering|Computer science|Artificial intelligence

COinS