Explore the vast range of titles available.

Background Joint modeling is appropriate when one wants to predict the time to an event with covariates that are measured longitudinally and are related to the event. An underlying random effects structure links the survival and longitudinal submodels and allows for individual-specific predictions. Multiple time-varying and time-invariant covariates can be included to potentially increase prediction accuracy. The goal of this study was to estimate a multivariate joint model on several longitudinal observational studies of Huntington’s disease, examine external validity performance, and compute individual-specific predictions for characterizing disease progression. Emphasis was on the survival submodel for predicting the hazard of motor diagnosis. Methods Data from four observational studies was analyzed: Enroll-HD, PREDICT-HD, REGISTRY, and Track-HD. A Bayesian approach to estimation was adopted, and external validation was performed using a time-varying AUC measure. Individual-specific cumulative hazard predictions were computed based on a simulation approach. The cumulative hazard was used for computing predicted age of motor onset and also for a deviance residual indicating the discrepancy between observed diagnosis status and model-based status. Results The joint model trained in a single study had very good performance in discriminating among diagnosed and pre-diagnosed participants in the remaining test studies, with the 5-year mean AUC = .83 (range .77–.90), and the 10-year mean AUC = .86 (range .82–.92). Graphical analysis of the predicted age of motor diagnosis showed an expected strong relationship with the trinucleotide expansion that causes Huntington’s disease. Graphical analysis of the deviance-type residual revealed there were individuals who converted to a diagnosis despite having relatively low model-based risk, others who had not yet converted despite having relatively high risk, and the majority falling between the two extremes. Conclusions Joint modeling is an improvement over traditional survival modeling because it considers all the longitudinal observations of covariates that are predictive of an event. Predictions from joint models can have greater accuracy because they are tailored to account for individual variability. These predictions can provide relatively accurate characterizations of individual disease progression, which might be important in the timing of interventions, determining the qualification for appropriate clinical trials, and general genotypic analysis.

Objective This study tested and validated an existing tool for its ability to predict the risk of long-term (ie, >6 weeks) sickness absence (LTSA) after four days of sick-listing. Methods A 9-item tool is completed online on the fourth day of sick-listing. The tool was tested in a sample (N=13 597) of food retail workers who reported sick between March and May 2017. It was validated in a new sample (N=104 698) of workers (83% retail) who reported sick between January 2020 and April 2021. LTSA risk predictions were calibrated with the Hosmer-Lemeshow (H-L) test; non-significant H-L P-values indicated adequate calibration. Discrimination between workers with and without LTSA was investigated with the area (AUC) under the receiver operating characteristic (ROC) curve. Results The data of 2203 (16%) workers in the test sample and 14 226 (13%) workers in the validation sample was available for analysis. In the test sample, the tool together with age and sex predicted LTSA (H-L test P=0.59) and discriminated between workers with and without LTSA [AUC 0.85, 95% confidence interval (CI) 0.83-0.87]. In the validation sample, LTSA risk predictions were adequate (H-L test P=0.13) and discrimination was excellent (AUC 0.91, 95% CI 0.90-0.92). The ROC curve had an optimal cut-off at a predicted 36% LTSA risk, with sensitivity 0.85 and specificity 0.83. Conclusion The existing 9-item tool can be used to invite sick-listed retail workers with a ≥36% LTSA risk for expedited consultations. Further studies are needed to determine LTSA cut-off risks for other economic sectors.

To derive and validate a set of computational models able to assess the risk of developing complications and experiencing adverse events for patients with diabetes. The models are developed on data from the Diabetes Control and Complications Trial (DCCT) and the Epidemiology of Diabetes Interventions and Complications (EDIC) studies, and are validated on an external, retrospectively collected cohort. We selected fifty-one clinical parameters measured at baseline during the DCCT as potential risk factors for the following adverse outcomes: Cardiovascular Diseases (CVD), Hypoglycemia, Ketoacidosis, Microalbuminuria, Proteinuria, Neuropathy and Retinopathy. For each outcome we applied a data-mining analysis protocol in order to identify the best-performing signature, i.e., the smallest set of clinical parameters that, considered jointly, are maximally predictive for the selected outcome. The predictive models built on the selected signatures underwent both an interval validation on the DCCT/EDIC data and an external validation on a retrospective cohort of 393 diabetes patients (49 Type I and 344 Type II) from the Chorleywood Medical Center, UK. The selected predictive signatures contain five to fifteen risk factors, depending on the specific outcome. Internal validation performances, as measured by the Concordance Index (CI), range from 0.62 to 0.83, indicating good predictive power. The models achieved comparable performances for the Type I and, quite surprisingly, Type II external cohort. Data-mining analyses of the DCCT/EDIC data allow the identification of accurate predictive models for diabetes-related complications. We also present initial evidences that these models can be applied on a more recent, European population.

Prognostic models that aim to improve the prediction of clinical events, individualized treatment and decision-making are increasingly being developed and published. However, relatively few models are externally validated and validation by independent researchers is rare. External validation is necessary to determine a prediction model’s reproducibility and generalizability to new and different patients. Various methodological considerations are important when assessing or designing an external validation study. In this article, an overview is provided of these considerations, starting with what external validation is, what types of external validation can be distinguished and why such studies are a crucial step towards the clinical implementation of accurate prediction models. Statistical analyses and interpretation of external validation results are reviewed in an intuitive manner and considerations for selecting an appropriate existing prediction model and external validation population are discussed. This study enables clinicians and researchers to gain a deeper understanding of how to interpret model validation results and how to translate these results to their own patient population.

Background Clinical prediction models should be validated before implementation in clinical practice. But is favorable performance at internal validation or one external validation sufficient to claim that a prediction model works well in the intended clinical context? Main body We argue to the contrary because (1) patient populations vary, (2) measurement procedures vary, and (3) populations and measurements change over time. Hence, we have to expect heterogeneity in model performance between locations and settings, and across time. It follows that prediction models are never truly validated. This does not imply that validation is not important. Rather, the current focus on developing new models should shift to a focus on more extensive, well-conducted, and well-reported validation studies of promising models. Conclusion Principled validation strategies are needed to understand and quantify heterogeneity, monitor performance over time, and update prediction models when appropriate. Such strategies will help to ensure that prediction models stay up-to-date and safe to support clinical decision-making.

1. Information criteria (ICs) are used widely for data summary and model building in ecology, especially in applied ecology and wildlife management. Although ICs are useful for distinguishing among rival candidate models, ICs do not necessarily indicate whether the \"best\" model (or a model-averaged version) is a good representation of the data or whether the model has useful \"explanatory\" or \"predictive\" ability. 2. As editors and reviewers, we have seen many submissions that did not evaluate whether the nominal \"best\" model(s) found using IC is a useful model in the above sense. 3. We scrutinized six leading ecological journals for papers that used IC to models. More than half of papers using IC for model comparison did not evaluate the adequacy of the best model(s) in either \"explaining\" or \"prdicting\" the data. 4. Synthesis and applications. Authors need to evaluate the adequacy of the model identified as the \"best\" model by using information criteria methods to provide convincing evidence to readers and users that inferences from the best models are useful and reliable.

Calibrated risk models are vital for valid decision support. We define four levels of calibration and describe implications for model development and external validation of predictions. We present results based on simulated data sets. A common definition of calibration is “having an event rate of R% among patients with a predicted risk of R%,” which we refer to as “moderate calibration.” Weaker forms of calibration only require the average predicted risk (mean calibration) or the average prediction effects (weak calibration) to be correct. “Strong calibration” requires that the event rate equals the predicted risk for every covariate pattern. This implies that the model is fully correct for the validation setting. We argue that this is unrealistic: the model type may be incorrect, the linear predictor is only asymptotically unbiased, and all nonlinear and interaction effects should be correctly modeled. In addition, we prove that moderate calibration guarantees nonharmful decision making. Finally, results indicate that a flexible assessment of calibration in small validation data sets is problematic. Strong calibration is desirable for individualized decision support but unrealistic and counter productive by stimulating the development of overly complex models. Model development and external validation should focus on moderate calibration.

ObjectiveThis study aimed to assess the association between routine cerebrospinal fluid (CSF) biochemical parameters and metagenomic next-generation sequencing (mNGS) results, and to develop a predictive model to optimize mNGS testing strategies in patients with suspected central nervous system (CNS) infections.MethodsWe retrospectively enrolled 110 patients with suspected CNS infections between December 2019 and January 2024. All underwent both CSF analysis and mNGS testing. Patients were divided into mNGS-positive (n = 62) and negative (n = 48) groups. Logistic regression identified independent predictors, and a nomogram was constructed based on CSF cell count and protein concentration. Model performance was assessed via receiver operating characteristic (ROC) curves, calibration plots, and decision curve analysis (DCA). Internal validation included 10-fold cross-validation and 1000-sample bootstrap. An external validation was performed using a cohort of 40 patients enrolled from another hospital campus (May–October 2024). The derivation cohort was retrospectively collected, whereas the external validation cohort was prospectively enrolled.ResultsmNGS positivity rate was 56.36%, significantly higher than CSF culture (6.36%), with an overall diagnostic concordance of 79.09%. Compared to the mNGS-negative group, positive patients had significantly higher CSF cell counts, protein levels, turbidity, ICU admission (ICUA), antimicrobial regimen adjustment (AAR), and mortality, while glucose was significantly lower (P < 0.05). Logistic regression confirmed CSF cell count binary variables (BV) and protein-BV as independent predictors (P < 0.05). The areas under curve (AUCs) for the cell-count, protein-only, and combined models were 0.827, 0.813, and 0.782, respectively. Internal validation showed stable results: 10-fold CV AUC = 0.773 ± 0.184 (95% CI: 0.641–0.904), bootstrap AUC = 0.770 ± 0.064 (95% CI: 0.766–0.774). External validation yielded an AUC of 0.763 (95% CI: 0.554–0.918), with sensitivity and specificity of 77.8% and 67.7%. Calibration and DCA demonstrated good agreement and clinical utility.ConclusionCSF cell count and protein are reliable predictors of mNGS positivity. The model for practice showed consistent diagnostic performance and may aid in guiding precision mNGS testing, particularly in resource-constrained settings.

Clinical prediction models provide risks of health outcomes that can inform patients and support medical decisions. However, most models never make it to actual implementation in practice. A commonly heard reason for this lack of implementation is that prediction models are often not externally validated. While we generally encourage external validation, we argue that an external validation is often neither sufficient nor required as an essential step before implementation. As such, any available external validation should not be perceived as a license for model implementation. We clarify this argument by discussing 3 common misconceptions about external validation. We argue that there is not one type of recommended validation design, not always a necessity for external validation, and sometimes a need for multiple external validations. The insights from this paper can help readers to consider, design, interpret, and appreciate external validation studies.

To evaluate how often newly developed risk prediction models undergo external validation and how well they perform in such validations. We reviewed derivation studies of newly proposed risk models and their subsequent external validations. Study characteristics, outcome(s), and models' discriminatory performance [area under the curve, (AUC)] in derivation and validation studies were extracted. We estimated the probability of having a validation, change in discriminatory performance with more stringent external validation by overlapping or different authors compared to the derivation estimates. We evaluated 127 new prediction models. Of those, for 32 models (25%), at least an external validation study was identified; in 22 models (17%), the validation had been done by entirely different authors. The probability of having an external validation by different authors within 5 years was 16%. AUC estimates significantly decreased during external validation vs. the derivation study [median AUC change: −0.05 (P < 0.001) overall; −0.04 (P = 0.009) for validation by overlapping authors; −0.05 (P < 0.001) for validation by different authors]. On external validation, AUC decreased by at least 0.03 in 19 models and never increased by at least 0.03 (P < 0.001). External independent validation of predictive models in different studies is uncommon. Predictive performance may worsen substantially on external validation.