Nonlinear and self-adapting methods for prediction

William Hsu, Purdue University

Abstract

The theme of this thesis is to develop techniques and methodologies used for (1) extracting features for prediction and (2) learning the relationships between these features and the quantity to be predicted. The results of these two techniques can be used to predict the temporal continuation of a time series. Through the use of the proposed clustering algorithm, ClusNet algorithm results in savings of both memory and prediction time required compared to the instance-based algorithms while retaining much of the advantages of the instance-based approach. The prediction achieved with ClusNet is comparable to other much more computational expensive algorithms. The performance of ClusNet is also relatively independent of the clustering algorithms used. We provide proofs of two clustering algorithms for ClusNet. We also analyzed the amount of space/time reduction possible using ClusNet compared to instance-based algorithms. The ClusNet algorithm is unsupervised. We proposed a novel supervised clustering scheme SupNet where the state vectors X are first clustered with respect to their outcomes Y and then each cluster can be subsequently clustered with respect to X. By analyzing the distribution of each component of state vectors X within each group of clusters, we propose several feature extracting and elimination algorithms for improving prediction. When the time series is chaotic in nature, Takens' Theorem proved that the features needed to predict the time series is contained in its embedding. We proposed a data inconsistency criterion and a systematic procedure to search for an embedding $\Pi$ that minimizes the criterion. This procedure is able to locate the embeddings used by researchers and find new embeddings for the Mackey-Glass time series. When chaotic assumptions cannot be made, predictive features can be extracted using a linear transformation computed from the decision boundary matrix. This transformation maps the original feature space to a lower dimensional feature space while at the same time preserving the decision boundary. Finally, in applications where some subsets of the given feature vectors may be confuse the predictor, we propose two feature subset selection algorithms to remove these confusing features. We demonstrate the algorithms on the German Mark versus the U.S. Dollar exchange rate.

Degree

Ph.D.

Advisors

Tenorio, Purdue University.

Subject Area

Electrical engineering

COinS