Journal Article

Bootstrap Model Selection

Shao, Jun

1996

Overview

In a regression problem, typically there are p explanatory variables possibly related to a response variable, and we wish to select a subset of the p explanatory variables to fit a model between these variables and the response. A bootstrap variable/model selection procedure is to select the subset of variables by minimizing bootstrap estimates of the prediction error, where the bootstrap estimates are constructed based on a data set of size n. Although the bootstrap estimates have good properties, this bootstrap selection procedure is inconsistent in the sense that the probability of selecting the optimal subset of variables does not converge to 1 as n → ∞. This inconsistency can be rectified by modifying the sampling method used in drawing bootstrap observations. For bootstrapping pairs (response, explanatory variable), it is found that instead of drawing n bootstrap observations (a customary bootstrap sampling plan), much less bootstrap observations should be sampled: The bootstrap selection procedure becomes consistent if we draw m bootstrap observations with m → ∞ and m/n → 0. For bootstrapping residuals, we modify the bootstrap sampling procedure by increasing the variability among the bootstrap observations. The consistency of the modified bootstrap selection procedures is established in various situations, including linear models, nonlinear models, generalized linear models, and autoregressive time series. The choice of the bootstrap sample size m and some computational issues are also discussed. Some empirical results are presented.

• Publisher: Taylor & Francis Group,American Statistical Association,Taylor & Francis Ltd
• Subject: Autoregressive time series / Bootstrap method / Bootstrap resampling / Bootstrap sample size / Bootstrapping / Consistent estimators / Errors / Estimators / Exact sciences and technology / Generalized linear model / Generalized linear models / Inconsistency / Inference / Inference from stochastic processes; time series analysis / Linear analysis / Linear inference, regression / Linear models / Linear regression / Mathematics / Modeling / Nonlinear analysis / Nonlinear regression / Prediction error / Probability and statistics / Property / Regression analysis / Sample size / Sampling / Sciences and techniques of general use / Selection procedures / Statistics / Theory and Methods / Time series / Variables