Journal Article

Comparison of statistical methods used to meta-analyse results from interrupted time series studies: an empirical study

Karahalios, Amalia, McKenzie, Joanne E., Korevaar, Elizabeth, Turner, Simon L., Taljaard, Monica, Forbes, Andrew B.

2024

Overview

Background The Interrupted Time Series (ITS) is a robust design for evaluating public health and policy interventions or exposures when randomisation may be infeasible. Several statistical methods are available for the analysis and meta-analysis of ITS studies. We sought to empirically compare available methods when applied to real-world ITS data. Methods We sourced ITS data from published meta-analyses to create an online data repository. Each dataset was re-analysed using two ITS estimation methods. The level- and slope-change effect estimates (and standard errors) were calculated and combined using fixed-effect and four random-effects meta-analysis methods. We examined differences in meta-analytic level- and slope-change estimates, their 95% confidence intervals, p-values, and estimates of heterogeneity across the statistical methods. Results Of 40 eligible meta-analyses, data from 17 meta-analyses including 282 ITS studies were obtained (predominantly investigating the effects of public health interruptions (88%)) and analysed. We found that on average , the meta-analytic effect estimates, their standard errors and between-study variances were not sensitive to meta-analysis method choice, irrespective of the ITS analysis method. However, across ITS analysis methods, for any given meta-analysis, there could be small to moderate differences in meta-analytic effect estimates, and important differences in the meta-analytic standard errors. Furthermore, the confidence interval widths and p -values for the meta-analytic effect estimates varied depending on the choice of confidence interval method and ITS analysis method. Conclusions Our empirical study showed that meta-analysis effect estimates, their standard errors, confidence interval widths and p -values can be affected by statistical method choice. These differences may importantly impact interpretations and conclusions of a meta-analysis and suggest that the statistical methods are not interchangeable in practice.

• Publisher: BioMed Central,BioMed Central Ltd,Springer Nature B.V,BMC
• Subject: Comparative analysis / Empirical study / Estimates / Evaluation / Health Sciences / Humans / Interrupted time series / Interrupted Time Series Analysis / Intervention / Medical research / Medicine / Medicine & Public Health / Medicine, Experimental / Meta-analysis / Methods / Methods and Applications for Real World Data / Public Health / Segmented regression / Simulation / Statistical methods / Statistical Theory and Methods / Statistics for Life Sciences / Theory of Medicine/Bioethics / Time series / Time-series analysis / Trends