Journal Article

Optimal Weight Choice for Frequentist Model Average Estimators

Zhang, Xinyu, Zou, Guohua, Wan, Alan T. K., Liang, Hua

2011

Overview

There has been increasing interest recently in model averaging within the frequentist paradigm. The main benefit of model averaging over model selection is that it incorporates rather than ignores the uncertainty inherent in the model selection process. One of the most important, yet challenging, aspects of model averaging is how to optimally combine estimates from different models. In this work, we suggest a procedure of weight choice for frequentist model average estimators that exhibits optimality properties with respect to the estimator's mean squared error (MSE). As a basis for demonstrating our idea, we consider averaging over a sequence of linear regression models. Building on this base, we develop a model weighting mechanism that involves minimizing the trace of an unbiased estimator of the model average estimator's MSE. We further obtain results that reflect the finite sample as well as asymptotic optimality of the proposed mechanism. A Monte Carlo study based on simulated and real data evaluates and compares the finite sample properties of this mechanism with those of existing methods. The extension of the proposed weight selection scheme to general likelihood models is also considered. This article has supplementary material online.

• Publisher: Taylor & Francis,American Statistical Association,Taylor & Francis Ltd
• Subject: Applications / Asymptotic optimality / Averages / Bias / Estimation / Estimation methods / Estimation theory / Estimators / Exact sciences and technology / Finite sample properties / Frequentism / General topics / Geometry / Least squares / Linear inference, regression / Linear models / Linear regression / Mallows criterion / Mathematical models / Mathematics / Maximum likelihood method / Measurement / Methodology / Modeling / Monte Carlo simulation / Parametric models / Probability and statistics / Property / Regression analysis / Sciences and techniques of general use / Smoothed AIC / Smoothed BIC / Statistics / Theory and Methods / Unbiased estimators / Unbiased MSE estimate / Uncertainty / Weighting