Retrospective-approximation algorithms for the multidimensional stochastic root -finding problem
The stochastic root-finding problem (SRFP) is that of solving a non-linear system of equations using only an oracle that provides estimates of the constituent functions at requested points. In 1994, Chen and Schmeiser developed a family of retrospective approximation (RA) algorithms that solve single-dimensional SRFPs efficiently. Fundamental to their approach is the concept of a sample-path approximation originally used in the context of stochastic optimization. The algorithms proceed by generating and solving with increasing accuracy, a sequence of approximate deterministic root-finding problems until a desired precision is achieved. This dissertation generalizes Chen and Schmeiser's RA algorithms to multiple dimensions. Generalizing Chen and Schmeiser's algorithms is non-trivial due to the complexities that naturally arise in designing algorithms in multiple dimensions, and more specifically due to difficulties in extending the mathematical machinery created by Chen and Schmeiser for the single-dimensional context. The primary contribution of this dissertation is developing Bounding RA, a sub-family of RA algorithms that work by identifying a sequence of polytopes that progressively decrease in size while approaching the solution. We prove that Bounding RA converges with probability one (w.p.1) on a class of functions obtained through a multidimensional generalization of strictly increasing functions. Efficiency results from (i) the RA structure, (ii) the idea of using bounding polytopes in solving sample-path problems, and (iii) careful step-size and direction choice during algorithm evolution. Empirical tests show that Bounding RA has good finite-time performance that is robust with respect to the location of the initial solution and algorithm-parameter values. Empirical tests also suggest that Bounding RA significantly outperforms Simultaneous Perturbation Stochastic Approximation (SPSA), which is currently the best-known algorithm for solving SRFPs.
Schmeiser, Purdue University.
Off-Campus Purdue Users:
To access this dissertation, please log in to our