Journal Article

BAYES AND EMPIRICAL-BAYES MULTIPLICITY ADJUSTMENT IN THE VARIABLE-SELECTION PROBLEM

Berger, James O., Scott, James G.

2010

Overview

This paper studies the multiplicity-correction effect of standard Bayesian variable-selection priors in linear regression. Our first goal is to clarify when, and how, multiplicity correction happens automatically in Bayesian analysis, and to distinguish this correction from the Bayesian Ockham's-razor effect. Our second goal is to contrast empirical-Bayes and fully Bayesian approaches to variable selection through examples, theoretical results and simulations. Considerable differences between the two approaches are found. In particular, we prove a theorem that characterizes a surprising aymptotic discrepancy between fully Bayes and empirical Bayes. This discrepancy arises from a different source than the failure to account for hyperparameter uncertainty in the empirical-Bayes estimate. Indeed, even at the extreme, when the empirical-Bayes estimate converges asymptotically to the true variable-inclusion probability, the potential for a serious difference remains.

• Publisher: Institute of Mathematical Statistics,The Institute of Mathematical Statistics
• Subject: 62J05 / 62J15 / Algebra / Approximation / Bayesian analysis / Bayesian model selection / Bayesian networks / Commutative rings and algebras / Datasets / Decision theory / empirical Bayes / Empiricism / Exact sciences and technology / General topics / Generalized linear models / Linear inference, regression / Mathematics / Maximum likelihood estimation / Modeling / multiple testing / Oracles / Parameter estimation / Parametric models / Probabilities / Probability and statistics / Regression analysis / Sciences and techniques of general use / Selection / Statistics / Studies / variable selection / Variables