Variances and quantiles in dynamic system performance: Point estimation and standard errors

Demet Ceylan Wood, Purdue University

Abstract

Simulation output analysis involves two major problems: point estimation and standard-error estimation. The first problem is choosing or developing methods to estimate system performance measures. The second problem is choosing or developing methods to estimate the standard deviations of the point estimators. Point estimation is the purpose of a simulation experiment; standard-error estimation indicates the point estimators' quality. Both problems have been studied extensively for means. This research addresses both problems for variances and for quantiles, in the context of steady-state autocorrelated time-series data such as arise in the simulation of dynamic stochastic systems. In point estimation we study estimators of the marginal variance. Interlaced variance estimators, a generalization of the classical sample variance, were recently proposed by other authors. We show that, although biased, the sample variance is mean-square-error (mse) optimal among interlaced variance estimators. In addition, we show that the sample variance with a modified denominator is the mse-optimal variance estimator within the larger family of quadratic-form estimators. In standard-error estimation we study overlapping batch statistics (OBS) as an estimator of the standard error of the sample variance (OBV) and of sample quantiles (OBQ). After proposing modifications to the original definitions of OBV and OBQ, we study their asymptotic and finite-sample properties. We show that both OBV and OBQ have desirable asymptotic properties, such as bias and variance diminishing to zero. We also show that OBV has desirable finite-sample properties, such as bias and variance being monotonic functions of the batch size. This monotonicity yields an mse-optimal batch-size formula for OBV. In contrast, we find that OBQ bias, variance and mse cycle as a function of the batch size. Furthermore, the mse-optimal batch size within a cycle depends on the marginal distribution, the quantile value and the point estimator of the quantile. This cyclic behavior complicates the analysis of the effect of the batch size, even for independently and identically distributed data. The cycle effect diminishes as the batch size increases, however, so the bias, variance and mse have tractable limiting behavior.

Degree

Ph.D.

Advisors

Schmeiser, Purdue University.

Subject Area

Industrial engineering

Off-Campus Purdue Users:
To access this dissertation, please log in to our
proxy server
.

Share

COinS