Journal Article

Calibration: the Achilles heel of predictive analytics

van Smeden, Maarten, McLernon, David J., Van Calster, Ben, Wynants, Laure, Steyerberg, Ewout W.

2019

Overview

Background The assessment of calibration performance of risk prediction models based on regression or more flexible machine learning algorithms receives little attention. Main text Herein, we argue that this needs to change immediately because poorly calibrated algorithms can be misleading and potentially harmful for clinical decision-making. We summarize how to avoid poor calibration at algorithm development and how to assess calibration at algorithm validation, emphasizing balance between model complexity and the available sample size. At external validation, calibration curves require sufficiently large samples. Algorithm updating should be considered for appropriate support of clinical practice. Conclusion Efforts are required to avoid poor calibration when developing prediction models, to evaluate calibration when validating models, and to update models when indicated. The ultimate aim is to optimize the utility of predictive analytics for shared decision-making and patient counseling.

• Publisher: BioMed Central,BioMed Central Ltd,Springer Nature B.V,BMC
• Subject: Adult / Aged / Algorithms / Analysis / Artificial intelligence / Biomedicine / Calibration / Calibration - standards / Data mining / Decision analysis / Decision making / Embryos / Estimates / Health care policy / Health counseling / Heterogeneity / Humans / In vitro fertilization / Learning algorithms / Machine learning / Machine Learning - standards / Male / Medicine / Medicine & Public Health / Middle Aged / Model performance / Opinion / Overfitting / Patients / Prediction models / Predictive analytics / Predictive Value of Tests / Regression analysis / Risk prediction models / Shared decision making