Optimizing Reflected Brownian Motion: A Numerical Study

Zihe Zhou, Purdue University

Abstract

Reflected Brownian motion is a canonical stochastic process used to model many engineered systems. For example, the state of a queueing system experiencing heavy traffic is well approximated by a reflected Brownian motion. It has also been used to model chemical reaction networks, as well as financial markets. The optimization and design of such systems can be modeled by stochastic optimization problems defined as additive functionals of reflected Brownian motions over a fixed time horizon. In this thesis, we are interested in a sub-class of problems where the design variable is the drift function of the RBM; in the one-dimensional setting we consider, this is without loss of generality. We also draw further distinctions with the stochastic optimal control problems that are driven by reflected Brownian motion processes, where the objective is to find an adaptive control policy. In contrast, the optimization problem here must be solved once at time zero and hence is not a stochastic optimal control problem. Modulo certain regularity conditions, our problem can be viewed as a deterministic optimal control problem that is (in theory) amenable to a dynamic programming solution. We derive the corresponding Hamilton-Jacobi-Bellman equation. However, this partial differential equation is both non-linear and with non-trivial boundary conditions, necessitating numerical solutions. We demonstrate numerical results from solving the Hamilton-Jacobi-Bellman equation using the finite element method. However, this approach suffers from the “curse of dimensionality.” Therefore we develop Monte Carlo simulation optimization methods for solving the stochastic optimization problem, for a time-discretized approximation of the original problem. We avoid the curse of dimensionality by using gradient descent to compute the optimal (time-discretized) drift. However, the gradient of the objective is not known in closed form and must be estimated using the simulation sample paths. We develop a bespoke gradient estimator that exploits the strong Markov property of the reflected Brownian motion process to express the gradient as a nested expectation. We compare the corresponding Monte Carlo estimator with the well-studied simultaneous perturbation stochastic approximation method. Our numerical results show that the Monte Carlo gradient descent method outperforms the simultaneous perturbation stochastic approximation method on our specific problem instance.

Degree

M.Sc.

Advisors

Honnappa, Purdue University.

Subject Area

Industrial engineering

COinS