Adaptive learning strategies in a time-series prediction task

In Jae Myung, Purdue University

Abstract

A vast majority of past research on decision making under uncertainty has focused on static and single-trial decisions while ignoring an important issue of learning. Even the research in dynamic decision making quite often concentrated only on the subjects' asymptote performance, thus leaving unexplored the question of how they achieved the asymptote level. Taking seriously the adaptive nature of decision making, the present study investigated adaptive learning strategies that people use in a stochastic environment in order to achieve their goal of optimizing an objective function. A time-series prediction learning task of the stock market in which subjects were required to predict stock prices based on outcome feedback was employed. In each experiment a crucial experimental factor which would disclose the nature of adaptive learning strategies was manipulated. Experiment 1 investigated the effects of the nonstationarity of environment where outcome probabilities change over time. In particular, time-invariance of the human learning system was tested by introducing nonstationary shifts in probability at different points of time during training. The results indicated that the predictions by the time-invariant model were clearly violated with the observation of a slow down tendency of the subjects' adaptation to the nonstationary shift as the time of the shift was delayed. In Experiment 2, the effects of an objective function were explored using different shapes of symmetric and asymmetric loss functions. The subjects' trial-by-trial changes in prediction as well as their asymptote performance were analyzed by qualitative and quantitative methods. The delta rule, the gradient learning model, and the hill-climbing learning model were tested. The major finding was a strong tendency for the subjects to use a modified form of the delta rule in which learning rate depends upon not only time but also performance level. A small but reliable evidence for the hill-climbing learning and no support for the gradient learning were obtained. A two-stage learning model that assumes the modified delta rule in initial training and the hill-climbing learning in later training was proposed and verified through computer simulations.

Degree

Ph.D.

Advisors

Busemeyer, Purdue University.

Subject Area

Psychology|Experiments|Psychology|Educational psychology

COinS