Detection of rare events and rule extraction by neural networks and decision trees

Wooyoung Choe, Purdue University

Abstract

Sample stratification is a technique for making each class in a sample have equal influence on decision making. For classification with neural networks, it is often preferable to use a stratified sample including an equal number of examples from each class. However, it is not always possible to have a stratified sample because of unavailability of examples (referred to as rare event cases). For rare event detection, the thesis represents two sample stratification schemes. The first scheme stratifies a sample by adding up the weighted sum of the derivatives during the backward pass of training. The second scheme uses bootstrap aggregating. After training neural networks with multiple sets of bootstrapped examples of the rare event classes and subsampled examples of common event classes, multiple voting for classification is performed. The second part of the research is on developing rule extraction algorithms from neural networks. As a method of overcoming the common criticism of black box approach of neural networks, the thesis proposes rule extraction algorithms by neural networks and decision trees based on parallel self-organizing hierarchical neural network. The first system solves the limitation of default rules which do not provide any interpretation about the data. By using another neural network for the input data that cannot be covered by extracted rules from the stage, more rules are generated and they are helpful in decision-making. The second system includes decision trees in rule-extracting process and gives more efficient and accurate results. This adds explanation capability to neural networks.

Degree

Ph.D.

Advisors

Ersoy, Purdue University.

Subject Area

Electrical engineering

COinS