Analysis of stochastic approximation and related algorithms

I-Jeng Wang, Purdue University

Abstract

A stochastic approximation algorithm is a recursive procedure for estimating zeros or extremal points of an unknown function based on noisy measurements of the function. Since the pioneering work of Robbins and Monro, stochastic approximation and related algorithms have been applied to a wide variety of areas, including stochastic optimization, adaptive control, and signal processing. The asymptotic behavior of such recursive algorithms, especially their convergence, has been extensively studied. Most of the analyses impose probabilistic assumptions on the noise sequence. Although such an approach exploits the power of probability theory, it may result in unduly restrictive and sometimes unrealistic assumptions. Furthermore, some important aspects of the sample-path behavior of algorithms are usually difficult to explore in a probabilistic approach. In this thesis, we develop a deterministic asymptotic analysis of stochastic approximation and related algorithms. We study four known conditions on noise sequences for convergence. We prove that all four conditions are equivalent, and are both necessary and sufficient for convergence of stochastic approximation algorithms. The result implies that the four conditions cannot be further weakened. We also establish a form of equivalence between weighted averaging and stochastic approximation, and present a necessary and sufficient condition for convergence of averaged stochastic approximation. For stochastic approximation algorithms with randomized directions, we prove convergence via our sample-path analysis, and compare the performance of different schemes via their asymptotic distribution. Finally, we give simulation results of a heuristic design to illustrate the importance of finite-horizon analysis, and discuss possible approaches to the problem. Most of the analysis presented in this thesis are deterministic. We believe that our deterministic sample-path approach provides better insights into the behavior of stochastic approximation algorithms.

Degree

Ph.D.

Advisors

Chong, Purdue University.

Subject Area

Systems design|Operations research

COinS