Bayesian model selection for high -dimensional models with prediction error loss and 0–1 loss
Abstract
In this thesis we try to find the correct form of Bayes rule in different special situations. The first part deals with a scenario provided by Stone (1979), where the loss is 0-1 loss and the true model is very high dimensional. Use of Bayes Information Criterion (BIC), in a naive way, leads to inconsistency in this situation. We show that the source of inconsistency is a misinterpretation of sample size in this situation. A better criterion, that accommodates the high dimensional nature of the problem, is introduced and named Generalized BIC. High dimensional models provide a scenario where, according to Shao (1997), Akaike Information Criterion (AIC) should perform well. Also the loss function assumed there is squared error prediction loss. We find the Bayes rule in this situation and show asymptotic equivalence of AIC and the Bayes rule with a further constraint of using least squares estimates to make predictions. However, a full Bayes rule that is allowed to use Bayes estimates to make predictions is shown to be better than either, in terms of minimizing squared error prediction loss. In the next part of the thesis, all these results are applied to an example of nested sequence of orthogonal linear models taken from George and Foster (2000) and an example of polynomial regression taken from Shibata (1983). Different kinds of prior structure have been tried on these examples to achieve lower prediction error. The last part of the thesis discusses about an approximation recommended by Bernardo and Smith (1994) that has been cited as a justification for Geometric Intrinsic Bayes Factor. We show that the approximation and the quantity that we are trying to approximate differ by a O([special characters omitted]) term. Also a wide variety of Bayes factors are compared with respect to the nature of their penalty per parameter.
Degree
Ph.D.
Advisors
Berger, Purdue University.
Subject Area
Statistics
Off-Campus Purdue Users:
To access this dissertation, please log in to our
proxy server.