Explore the vast range of titles available.

Background Deep learning models have had a lot of success in various fields. However, on structured data they have struggled. Here we apply four state-of-the-art supervised deep learning models using the attention mechanism and compare against logistic regression and XGBoost using discrimination, calibration and clinical utility. Methods We develop the models using a general practitioners database. We implement a recurrent neural network, a transformer with and without reverse distillation and a graph neural network. We measure discrimination using the area under the receiver operating characteristic curve (AUC) and the area under the precision recall curve (AUPRC). We assess smooth calibration using restricted cubic splines and clinical utility with decision curve analysis. Results Our results show that deep learning approaches can improve discrimination up to 2.5% points AUC and 7.4% points AUPRC. However, on average the baselines are competitive. Most models are similarly calibrated as the baselines except for the graph neural network. The transformer using reverse distillation shows the best performance in clinical utility on two out of three prediction problems over most of the prediction thresholds. Conclusion In this study, we evaluated various approaches in supervised learning using neural networks and attention. Here we do a rigorous comparison, not only looking at discrimination but also calibration and clinical utility. There is value in using deep learning models on electronic health record data since it can improve discrimination and clinical utility while providing good calibration. However, good baseline methods are still competitive.

Background Many dementia prediction models have been developed, but only few have been externally validated, which hinders clinical uptake and may pose a risk if models are applied to actual patients regardless. Externally validating an existing prediction model is a difficult task, where we mostly rely on the completeness of model reporting in a published article. In this study, we aim to externally validate existing dementia prediction models. To that end, we define model reporting criteria, review published studies, and externally validate three well reported models using routinely collected health data from administrative claims and electronic health records. Methods We identified dementia prediction models that were developed between 2011 and 2020 and assessed if they could be externally validated given a set of model criteria. In addition, we externally validated three of these models (Walters’ Dementia Risk Score, Mehta’s RxDx-Dementia Risk Index, and Nori’s ADRD dementia prediction model) on a network of six observational health databases from the United States, United Kingdom, Germany and the Netherlands, including the original development databases of the models. Results We reviewed 59 dementia prediction models. All models reported the prediction method, development database, and target and outcome definitions. Less frequently reported by these 59 prediction models were predictor definitions (52 models) including the time window in which a predictor is assessed (21 models), predictor coefficients (20 models), and the time-at-risk (42 models). The validation of the model by Walters (development c-statistic: 0.84) showed moderate transportability (0.67–0.76 c-statistic). The Mehta model (development c-statistic: 0.81) transported well to some of the external databases (0.69–0.79 c-statistic). The Nori model (development AUROC: 0.69) transported well (0.62–0.68 AUROC) but performed modestly overall. Recalibration showed improvements for the Walters and Nori models, while recalibration could not be assessed for the Mehta model due to unreported baseline hazard. Conclusion We observed that reporting is mostly insufficient to fully externally validate published dementia prediction models, and therefore, it is uncertain how well these models would work in other clinical settings. We emphasize the importance of following established guidelines for reporting clinical prediction models. We recommend that reporting should be more explicit and have external validation in mind if the model is meant to be applied in different settings.

BackgroundThere is currently no consensus on the impact of class imbalance methods on the performance of clinical prediction models. We aimed to empirically investigate the impact of random oversampling and random undersampling, two commonly used class imbalance methods, on the internal and external validation performance of prediction models developed using observational health data.MethodsWe developed and externally validated prediction models for various outcomes of interest within a target population of people with pharmaceutically treated depression across four large observational health databases. We used three different classifiers (lasso logistic regression, random forest, XGBoost) and varied the target imbalance ratio. We evaluated the impact on model performance in terms of discrimination and calibration. Discrimination was assessed using the area under the receiver operating characteristic curve (AUROC) and calibration was assessed using calibration plots.ResultsWe developed and externally validated a total of 1,566 prediction models. On internal and external validation, random oversampling and random undersampling generally did not result in higher AUROCs. Moreover, we found overestimated risks, although this miscalibration could largely be corrected by recalibrating the models towards the imbalance ratios in the original dataset.ConclusionsOverall, we found that random oversampling or random undersampling generally does not improve the internal and external validation performance of prediction models developed in large observational health databases. Based on our findings, we do not recommend applying random oversampling or random undersampling when developing prediction models in large observational health databases.

Background A prediction model can be a useful tool to quantify the risk of a patient developing dementia in the next years and take risk-factor-targeted intervention. Numerous dementia prediction models have been developed, but few have been externally validated, likely limiting their clinical uptake. In our previous work, we had limited success in externally validating some of these existing models due to inadequate reporting. As a result, we are compelled to develop and externally validate novel models to predict dementia in the general population across a network of observational databases. We assess regularization methods to obtain parsimonious models that are of lower complexity and easier to implement. Methods Logistic regression models were developed across a network of five observational databases with electronic health records (EHRs) and claims data to predict 5-year dementia risk in persons aged 55–84. The regularization methods L1 and Broken Adaptive Ridge (BAR) as well as three candidate predictor sets to optimize prediction performance were assessed. The predictor sets include a baseline set using only age and sex, a full set including all available candidate predictors, and a phenotype set which includes a limited number of clinically relevant predictors. Results BAR can be used for variable selection, outperforming L1 when a parsimonious model is desired. Adding candidate predictors for disease diagnosis and drug exposure generally improves the performance of baseline models using only age and sex. While a model trained on German EHR data saw an increase in AUROC from 0.74 to 0.83 with additional predictors, a model trained on US EHR data showed only minimal improvement from 0.79 to 0.81 AUROC. Nevertheless, the latter model developed using BAR regularization on the clinically relevant predictor set was ultimately chosen as best performing model as it demonstrated more consistent external validation performance and improved calibration. Conclusions We developed and externally validated patient-level models to predict dementia. Our results show that although dementia prediction is highly driven by demographic age, adding predictors based on condition diagnoses and drug exposures further improves prediction performance. BAR regularization outperforms L1 regularization to yield the most parsimonious yet still well-performing prediction model for dementia.

BackgroundPrognostic models help aid medical decision-making. Various prognostic models are available via websites such as MDCalc, but these models typically predict one outcome, for example, stroke risk. Each model requires individual predictors, for example, age, lab results and comorbidities. There is no clinical tool available to predict multiple outcomes from a list of common medical predictors.ObjectiveIdentify a constrained set of outcome-agnostic predictors.MethodsWe proposed a novel technique aggregating the standardised mean difference across hundreds of outcomes to learn a constrained set of predictors that appear to be predictive of many outcomes. Model performance was evaluated using the constrained set of predictors across eight prediction tasks. We compared against existing models, models using only age/sex predictors and models without any predictor constraints.ResultsWe identified 67 predictors in our constrained set, plus age/sex. Our predictors included illnesses in the following categories: cardiovascular, kidney/liver, mental health, gastrointestinal, infectious and oncologic. Models developed using the constrained set of predictors achieved comparable discrimination compared with models using hundreds or thousands of predictors for five of the eight prediction tasks and slightly lower discrimination for three of the eight tasks. The constrained predictor models performed as good or better than all existing clinical models.ConclusionsIt is possible to develop models for hundreds or thousands of outcomes that use the same small set of predictors. This makes it feasible to implement many prediction models via a single website form. Our set of predictors can also be used for future models and prognostic model research.

Deep brain stimulation (DBS) is being investigated as a treatment for patients with refractory major depressive disorder (MDD). However, little is known about how DBS exerts its antidepressive effects. Here, we investigated whether ventral anterior limb of the internal capsule stimulation modulates a limbic network centered around the amygdala in patients with treatment-resistant MDD. Nine patients underwent resting state functional magnetic resonance scans before DBS surgery and after 1 year of treatment. In addition, they were scanned twice within 2 weeks during the subsequent double-blind cross-over phase with active and sham treatment. Twelve matched controls underwent scans at the same time intervals to account for test-retest effects. The imaging data were investigated with functional connectivity (FC) analysis and dynamic causal modelling. Results showed that 1 year of DBS treatment was associated with increased FC of the left amygdala with precentral cortex and left insula, along with decreased bilateral connectivity between nucleus accumbens and ventromedial prefrontal cortex. No changes in FC were observed during the cross-over phase. Effective connectivity analyses using dynamic causal models revealed widespread amygdala-centric changes between presurgery and 1 year follow-up, while the cross-over phase was associated with insula-centric changes between active and sham stimulation. These results suggest that ventral anterior limb of the internal capsule DBS results in complex rebalancing of the limbic network involved in emotion, reward, and interoceptive processing.

No diagnostic biomarkers are available for obsessive-compulsive disorder (OCD). Here, we aimed to identify magnetic resonance imaging (MRI) biomarkers for OCD, using 46 data sets with 2304 OCD patients and 2068 healthy controls from the ENIGMA consortium. We performed machine learning analysis of regional measures of cortical thickness, surface area and subcortical volume and tested classification performance using cross-validation. Classification performance for OCD vs. controls using the complete sample with different classifiers and cross-validation strategies was poor. When models were validated on data from other sites, model performance did not exceed chance-level. In contrast, fair classification performance was achieved when patients were grouped according to their medication status. These results indicate that medication use is associated with substantial differences in brain anatomy that are widely distributed, and indicate that clinical heterogeneity contributes to the poor performance of structural MRI as a disease marker.

ObjectiveDevelop and validate models that predict mortality of patients diagnosed with COVID-19 admitted to the hospital.DesignRetrospective cohort study.SettingA multicentre cohort across 10 Dutch hospitals including patients from 27 February to 8 June 2020.ParticipantsSARS-CoV-2 positive patients (age ≥18) admitted to the hospital.Main outcome measures21-day all-cause mortality evaluated by the area under the receiver operator curve (AUC), sensitivity, specificity, positive predictive value and negative predictive value. The predictive value of age was explored by comparison with age-based rules used in practice and by excluding age from the analysis.Results2273 patients were included, of whom 516 had died or discharged to palliative care within 21 days after admission. Five feature sets, including premorbid, clinical presentation and laboratory and radiology values, were derived from 80 features. Additionally, an Analysis of Variance (ANOVA)-based data-driven feature selection selected the 10 features with the highest F values: age, number of home medications, urea nitrogen, lactate dehydrogenase, albumin, oxygen saturation (%), oxygen saturation is measured on room air, oxygen saturation is measured on oxygen therapy, blood gas pH and history of chronic cardiac disease. A linear logistic regression and non-linear tree-based gradient boosting algorithm fitted the data with an AUC of 0.81 (95% CI 0.77 to 0.85) and 0.82 (0.79 to 0.85), respectively, using the 10 selected features. Both models outperformed age-based decision rules used in practice (AUC of 0.69, 0.65 to 0.74 for age >70). Furthermore, performance remained stable when excluding age as predictor (AUC of 0.78, 0.75 to 0.81).ConclusionBoth models showed good performance and had better test characteristics than age-based decision rules, using 10 admission features readily available in Dutch hospitals. The models hold promise to aid decision-making during a hospital bed shortage.

Background: Conventional prediction methods such as logistic regression and gradient boosting have been widely utilized for disease onset prediction for their reliability and interpretability. Deep learning methods promise enhanced prediction performance by extracting complex patterns from clinical data, but face challenges like data sparsity and high dimensionality. Methods: This study compares conventional and deep learning approaches to predict lung cancer, dementia, and bipolar disorder using observational data from eleven databases from North America, Europe, and Asia. Models were developed using logistic regression, gradient boosting, ResNet, and Transformer, and validated both internally and externally across the data sources. Discrimination performance was assessed using AUROC, and calibration was evaluated using Eavg. Findings: Across 11 datasets, conventional methods generally outperformed deep learning methods in terms of discrimination performance, particularly during external validation, highlighting their better transportability. Learning curves suggest that deep learning models require substantially larger datasets to reach the same performance levels as conventional methods. Calibration performance was also better for conventional methods, with ResNet showing the poorest calibration. Interpretation: Despite the potential of deep learning models to capture complex patterns in structured observational healthcare data, conventional models remain highly competitive for disease onset prediction, especially in scenarios involving smaller datasets and if lengthy training times need to be avoided. The study underscores the need for future research focused on optimizing deep learning models to handle the sparsity, high dimensionality, and heterogeneity inherent in healthcare datasets, and find new strategies to exploit the full capabilities of deep learning methods.

Neurosurgical interventions including deep brain stimulation (DBS) and capsulotomy have been demonstrated effective for refractory obsessive-compulsive disorder (OCD), although treatment-shared/-specific network mechanisms remain largely unclear. We retrospectively analyzed resting-state fMRI data from three cohorts: a cross-sectional dataset of 186 subjects (104 OCD and 82 healthy controls), and two longitudinal datasets of refractory patients receiving ventral capsule/ventral striatum DBS (14 OCD) and anterior capsulotomy (27 OCD). We developed a machine learning model predictive of OCD symptoms (indexed by the Yale–Brown Obsessive Compulsive Scale, Y-BOCS) based on functional connectivity profiles and used graphic measures of network communication to characterize treatment-induced profile changes. We applied a linear model on 2 levels treatments (DBS or capsulotomy) and outcome to identify whether pre-surgical network communication was associated with differential treatment outcomes. We identified 54 functional connectivities within fronto-subcortical networks significantly predictive of Y-BOCS score in patients across 3 independent cohorts, and observed a coexisting pattern of downregulated cortico-subcortical and upregulated cortico-cortical network communication commonly shared by DBS and capsulotomy. Furthermore, increased cortico-cortical communication at ventrolateral and centrolateral prefrontal cortices induced by DBS and capsulotomy contributed to improvement of mood and anxiety symptoms, respectively (p < 0.05). Importantly, pretreatment communication of ventrolateral and centrolateral prefrontal cortices were differentially predictive of mood and anxiety improvements by DBS and capsulotomy (effect sizes = 0.45 and 0.41, respectively). These findings unravel treatment-shared and treatment-specific network characteristics induced by DBS and capsulotomy, which may facilitate the search of potential evidence-based markers for optimally selecting among treatment options for a patient.