SAMPLING CHARACTERISTICS OF MULTIVARIATE GENERALIZABILITY STATISTICS

DAVID GEORGE WARD, Purdue University

Abstract

This study was a Monte Carlo simulation of the sampling distributions of covariance matrices and characteristic roots and vectors in several balanced multivariate, multifacet designs using the random effects model. The distributions were examined as a function of the number of variables, the sample size, and the number of levels of the facets. The major focus was on the sampling characteristics of the weight vectors which transform the original variables into orthogonal, sequentially maximally generalizable composites. The vector which defines the most generalizable composite, i.e., the vector corresponding to the largest g-coefficient (characteristic root), was less stable than the vector corresponding to the smallest root, as measured by the cosine of the angle between the vector pairs over 300 independent samples. The first vector, however, was more stable than vectors corresponding to g-coefficients between the largest and smallest. In general, the elements of the first vector had smaller variance than elements of other vectors and the variance of elements decreased as eigenvalues increased and as sample size increased. For vectors in well-defined vector spaces, in terms of distinctness of eigenvalues, the larger the absolute magnitude of the elements, the more stable the vector across independent samples. Further, as the sample size increases, there is a proportional increase in stability and a reduction in variance. For example, a sample size of at least 20 times the number of variables is necessary to obtain adequate estimates of the weight vectors, defined by a congruence coefficient of at least .80. This study also examined the factor indeterminancy problem in the random effects model as used in generalizability theory. For univariate repeated measures, the minimum correlation between equivalent common factors was found to be related to the generalizability coefficient. In spite of effects and universe scores not being uniquely defined, the variance components associated with effects are determinate. The results indicate that the greater the generalizability of the variable over levels of the facet of interest, the less the indeterminacy, as measured by the minimum correlation between equivalent common factors. These results indicate consistency among several methods of assessing the reliability and dependability of measurement procedures.

Degree

Ph.D.

Subject Area

Psychology|Experiments

COinS