Explore the vast range of titles available.

Background Interrupted time series (ITS) studies are frequently used to evaluate the effects of population-level interventions or exposures. However, examination of the performance of statistical methods for this design has received relatively little attention. Methods We simulated continuous data to compare the performance of a set of statistical methods under a range of scenarios which included different level and slope changes, varying lengths of series and magnitudes of lag-1 autocorrelation. We also examined the performance of the Durbin-Watson (DW) test for detecting autocorrelation. Results All methods yielded unbiased estimates of the level and slope changes over all scenarios. The magnitude of autocorrelation was underestimated by all methods, however, restricted maximum likelihood (REML) yielded the least biased estimates. Underestimation of autocorrelation led to standard errors that were too small and coverage less than the nominal 95%. All methods performed better with longer time series, except for ordinary least squares (OLS) in the presence of autocorrelation and Newey-West for high values of autocorrelation. The DW test for the presence of autocorrelation performed poorly except for long series and large autocorrelation. Conclusions From the methods evaluated, OLS was the preferred method in series with fewer than 12 points, while in longer series, REML was preferred. The DW test should not be relied upon to detect autocorrelation, except when the series is long. Care is needed when interpreting results from all methods, given confidence intervals will generally be too narrow. Further research is required to develop better performing methods for ITS, especially for short series.

Background The Interrupted Time Series (ITS) is a quasi-experimental design commonly used in public health to evaluate the impact of interventions or exposures. Multiple statistical methods are available to analyse data from ITS studies, but no empirical investigation has examined how the different methods compare when applied to real-world datasets. Methods A random sample of 200 ITS studies identified in a previous methods review were included. Time series data from each of these studies was sought. Each dataset was re-analysed using six statistical methods. Point and confidence interval estimates for level and slope changes, standard errors, p- values and estimates of autocorrelation were compared between methods. Results From the 200 ITS studies, including 230 time series, 190 datasets were obtained. We found that the choice of statistical method can importantly affect the level and slope change point estimates, their standard errors, width of confidence intervals and p- values. Statistical significance (categorised at the 5% level) often differed across the pairwise comparisons of methods, ranging from 4 to 25% disagreement. Estimates of autocorrelation differed depending on the method used and the length of the series. Conclusions The choice of statistical method in ITS studies can lead to substantially different conclusions about the impact of the interruption. Pre-specification of the statistical method is encouraged, and naive conclusions based on statistical significance should be avoided.

Interrupted time series (ITS) designs are frequently used in public health to examine whether an intervention or exposure has influenced health outcomes. Few reviews have been undertaken to examine the design characteristics, statistical methods, and completeness of reporting of published ITS studies. We used stratified random sampling to identify 200 ITS studies that evaluated public health interventions or exposures from PubMed (2013–2017). Study characteristics, details of statistical models and estimation methods used, effect metrics, and parameter estimates were extracted. From the 200 studies, 230 time series were examined. Common statistical methods used were linear regression (31%, 72/230) and autoregressive integrated moving average (19%, 43/230). In 17% (40/230) of the series, we could not determine the statistical method used. Autocorrelation was acknowledged in 63% (145/230) of the series. An estimate of the autocorrelation coefficient was given for only 1% of the series (3/230). Measures of precision were reported for 63% of effect measures (541/852). Many aspects of the design, methods, analysis, and reporting of ITS studies can be improved, particularly description of the statistical methods and approaches to adjust for and estimate autocorrelation. More guidance on the conduct and reporting of ITS studies is needed to improve this study design.

Background Standard stepped wedge trials, where clusters switch from the control to the intervention condition in a staggered manner, can be costly and burdensome. Recent work has shown that the amount of information contributed by each cluster in each period differs, with some cluster-periods contributing a relatively small amount of information. We investigate the patterns of the information content of cluster-period cells upon iterative removal of low-information cells, assuming a model for continuous outcomes with constant cluster-period size, categorical time period effects, and exchangeable and discrete-time decay intracluster correlation structures. Methods We sequentially remove pairs of “centrosymmetric” cluster-period cells from an initially complete stepped wedge design which contribute the least amount of information to the estimation of the treatment effect. At each iteration, we update the information content of the remaining cells, determine the pair of cells with the lowest information content, and repeat this process until the treatment effect cannot be estimated. Results We demonstrate that as more cells are removed, more information is concentrated in the cells near the time of the treatment switch, and in “hot-spots” in the corners of the design. For the exchangeable correlation structure, removing the cells from these hot-spots leads to a marked reduction in study precision and power, however the impact of this is lessened for the discrete-time decay structure. Conclusions Removing cluster-period cells distant from the time of the treatment switch may not lead to large reductions in precision or power, implying that certain incomplete designs may be almost as powerful as complete designs.

Background Simulation is an important tool for assessing the performance of statistical methods for the analysis of data and for the planning of studies. While methods are available for the simulation of correlated binary random variables, all have significant practical limitations for simulating outcomes from longitudinal cluster randomised trial designs, such as the cluster randomised crossover and the stepped wedge trial designs. For these trial designs as the number of observations in each cluster increases these methods either become computationally infeasible or their range of allowable correlations rapidly shrinks to zero. Methods In this paper we present a simple method for simulating binary random variables with a specified vector of prevalences and correlation matrix. This method allows for the outcome prevalence to change due to treatment or over time, and for a ‘nested exchangeable’ correlation structure, in which observations in the same cluster are more highly correlated if they are measured in the same time period than in different time periods, and where different individuals are measured in each time period. This means that our method is also applicable to more general hierarchical clustered data contexts, such as students within classrooms within schools. The method is demonstrated by simulating 1000 datasets with parameters matching those derived from data from a cluster randomised crossover trial assessing two variants of stress ulcer prophylaxis. Results Our method is orders of magnitude faster than the most well known general simulation method while also allowing a much wider range of correlations than alternative methods. An implementation of our method is available in an R package NestBin . Conclusions This simulation method is the first to allow for practical and efficient simulation of large datasets of binary outcomes with the commonly used nested exchangeable correlation structure. This will allow for much more effective testing of designs and inference methods for longitudinal cluster randomised trials with binary outcomes.

Objective In the majority of studies, the effect of physical activity (PA) on cardiovascular disease (CVD) and mortality is estimated at a single time point. The impact of long-term PA is likely to differ. Our study objective was to estimate the effect of long-term adult-life PA compared with long-term inactivity on the risk of incident CVD, all-cause mortality and CVD-attributable mortality. Design Observational cohort study. Setting Framingham, MA, USA. Patients 4729 Framingham Heart Study participants who were alive and CVD-free in 1956. Exposures PA was measured at three visits over 30 years along with a variety of risk factors for CVD. Cumulative PA was defined as long-term active versus long-term inactive. Main outcome measures Incident CVD, all-cause mortality and CVD-attributable mortality. Results During 40 years of follow-up there were 2594 cases of incident CVD, 1313 CVD-attributable deaths and 3521 deaths. Compared with long-term physical inactivity, the rate ratio of long-term PA was 0.95 (95% CI 0.84 to 1.07) for CVD, 0.81 (0.71 to 0.93) for all-cause mortality and 0.83 (0.72 to 0.97) for CVD-attributable mortality. Assessment of effect modification by sex suggests greater protective effect of long-term PA on CVD incidence (p value for interaction=0.004) in men (0.79 (0.66 to 0.93)) than in women (1.15 (0.97 to 1.37)). Conclusions Cumulative long-term PA has a protective effect on incidence of all-cause and CVD-attributable mortality compared with long-term physical inactivity. In men, but not women, long-term PA also appears to have a protective effect on incidence of CVD.

Background Staircase designs are incomplete stepped wedge designs that, unlike standard stepped wedge designs, require clusters to contribute data for only a limited number of trial periods. Previous work has provided formulae based on asymptotic results for the calculation of the power of staircase designs to detect treatment effects of interest. Methods We conduct a simulation study to assess the finite sample performance of these formulae, and the impact of misspecifying the correlation structure when analysing data from staircase designs on inference for the treatment effect, under a range of realistic trial settings. This study focuses on basic staircase designs with one control period followed by one intervention period in each sequence. We simulate staircase trial datasets with continuous outcomes and a repeated cross-sectional measurement scheme under exchangeable and block-exchangeable intracluster correlation structures, and then fit linear mixed models with linear and categorical time period effects. For settings with a small number of clusters, Kenward-Roger and Satterthwaite small-sample corrections are applied. Comparisons are made between nominal and observed Type I error rates, and theoretically-derived study power and empirical power. The impact on inference for the treatment effect when misspecifying the intracluster correlation structure is assessed through considering performance metrics including bias and 95% confidence interval coverage. Results Data analysis assuming an exchangeable correlation structure and application of the Satterthwaite correction controls Type I error well when the correlation structure is correctly specified, and there are a sufficient number of clusters. For the true block-exchangeable model, when fitting the correct model with the Satterthwaite correction, the observed Type I error (empirical power) can be higher (lower) than the nominal (i.e., theoretical) value when there is only 1 cluster per sequence, but otherwise, it aligns well with the nominal (theoretical) value. Misspecification of the correlation structure (fitting an exchangeable model when the true structure is block-exchangeable) can lead to inflated Type I error and poor confidence interval coverage. Conclusions Staircase designs with one cluster per sequence should be used with caution. Additionally, using a correlation structure that allows for decay is preferable for making valid inferences for the estimation of the treatment effect.

Background Stepped wedge trials are an appealing and potentially powerful cluster randomized trial design. However, they are frequently implemented with a small number of clusters. Standard analysis methods for these trials such as a linear mixed model with estimation via maximum likelihood or restricted maximum likelihood (REML) rely on asymptotic properties and have been shown to yield inflated type I error when applied to studies with a small number of clusters. Small-sample methods such as the Kenward-Roger approximation in combination with REML can potentially improve estimation of the fixed effects such as the treatment effect. A Bayesian approach may also be promising for such multilevel models but has not yet seen much application in cluster randomized trials. Methods We conducted a simulation study comparing the performance of REML with and without a Kenward-Roger approximation to a Bayesian approach using weakly informative prior distributions on the intracluster correlation parameters. We considered a continuous outcome and a range of stepped wedge trial configurations with between 4 and 40 clusters. To assess method performance we calculated bias and mean squared error for the treatment effect and correlation parameters and the coverage of 95% confidence/credible intervals and relative percent error in model-based standard error for the treatment effect. Results Both REML with a Kenward-Roger standard error and degrees of freedom correction and the Bayesian method performed similarly well for the estimation of the treatment effect, while intracluster correlation parameter estimates obtained via the Bayesian method were less variable than REML estimates with different relative levels of bias. Conclusions The use of REML with a Kenward-Roger approximation may be sufficient for the analysis of stepped wedge cluster randomized trials with a small number of clusters. However, a Bayesian approach with weakly informative prior distributions on the intracluster correlation parameters offers a viable alternative, particularly when there is interest in the probability-based inferences permitted within this paradigm.

Objectives The interrupted time series (ITS) design is commonly used to investigate the impact of an intervention or exposure in public health. There are many statistical methods that can be used to analyse ITS data and to meta-analyse their results. We undertook two empirical studies to investigate: (i) how effect estimates (and associated statistics) compared when six statistical methods were applied to 190 real-world datasets; and (ii) how meta-analysis effect estimates (and associated statistics) compared when the combinations of two ITS analysis methods and five meta-analysis methods were applied to 17 real-world meta-analyses including 283 ITS datasets. Here we present a curated repository of a subset of ITS datasets from these studies. Data description The repository includes 430 ITS datasets curated from the two empirical studies. The datasets are diverse in the populations, interruptions and outcomes examined, and are methodologically diverse in the outcome types, aggregation time intervals, number of timepoints and segments. Most of the datasets are from public health. For each dataset, we provide the outcome value at each timepoint and the segment (indicating different interruptions), along with characteristics of the dataset. This repository may be of value for future research of ITS studies, and as a source of examples of ITS for use in teaching.

Background The Interrupted Time Series (ITS) is a robust design for evaluating public health and policy interventions or exposures when randomisation may be infeasible. Several statistical methods are available for the analysis and meta-analysis of ITS studies. We sought to empirically compare available methods when applied to real-world ITS data. Methods We sourced ITS data from published meta-analyses to create an online data repository. Each dataset was re-analysed using two ITS estimation methods. The level- and slope-change effect estimates (and standard errors) were calculated and combined using fixed-effect and four random-effects meta-analysis methods. We examined differences in meta-analytic level- and slope-change estimates, their 95% confidence intervals, p-values, and estimates of heterogeneity across the statistical methods. Results Of 40 eligible meta-analyses, data from 17 meta-analyses including 282 ITS studies were obtained (predominantly investigating the effects of public health interruptions (88%)) and analysed. We found that on average , the meta-analytic effect estimates, their standard errors and between-study variances were not sensitive to meta-analysis method choice, irrespective of the ITS analysis method. However, across ITS analysis methods, for any given meta-analysis, there could be small to moderate differences in meta-analytic effect estimates, and important differences in the meta-analytic standard errors. Furthermore, the confidence interval widths and p -values for the meta-analytic effect estimates varied depending on the choice of confidence interval method and ITS analysis method. Conclusions Our empirical study showed that meta-analysis effect estimates, their standard errors, confidence interval widths and p -values can be affected by statistical method choice. These differences may importantly impact interpretations and conclusions of a meta-analysis and suggest that the statistical methods are not interchangeable in practice.