Journal Article

Recovering Best Statistical Guarantees via the Empirical Divergence-Based Distributionally Robust Optimization

Lam, Henry

2019

Overview

Distributionally robust optimization (DRO), a recent methodology to handle stochastic optimization problems in the presence of data, is based on robustifications of stochastic constraints that are enforced to hold over suitably constructed sets of underlying probability distributions. Although DRO enjoys valid feasibility guarantees, it often leads to over-conservative solutions. The paper “Recovering best statistical guarantees via the empirical divergence-based distributionally robust optimization” by Lam studies a calibration method for distributional sets to combat conservativeness via a new interpretation of DRO through the statistical angle of empirical likelihood and empirical processes. The proposed method targets achieving precise confidence level guarantees that lead to superior performances over previous approaches. We investigate the use of distributionally robust optimization (DRO) as a tractable tool to recover the asymptotic statistical guarantees provided by the central limit theorem, for maintaining the feasibility of an expected value constraint under ambiguous probability distributions. We show that using empirically defined Burg-entropy divergence balls to construct the DRO can attain such guarantees. These balls, however, are not reasoned from the standard data-driven DRO framework because, by themselves, they can have low or even zero probability of covering the true distribution. Rather, their superior statistical performances are endowed by linking the resulting DRO with empirical likelihood and empirical processes. We show that the sizes of these balls can be optimally calibrated using χ 2 -process excursion. We conduct numerical experiments to support our theoretical findings.

• Publisher: INFORMS,Institute for Operations Research and the Management Sciences
• Subject: Analysis / Asymptotic methods / central limit theorem / chi-square process / distributionally robust optimization / Divergence / empirical likelihood / empirical process / Entropy / METHODS / Operations research / Optimization / Optimization theory / Probability / Probability distributions / Production management / Robustness (mathematics) / Standard data / Statistical analysis