Explore the vast range of titles available.

Background Many dementia prediction models have been developed, but only few have been externally validated, which hinders clinical uptake and may pose a risk if models are applied to actual patients regardless. Externally validating an existing prediction model is a difficult task, where we mostly rely on the completeness of model reporting in a published article. In this study, we aim to externally validate existing dementia prediction models. To that end, we define model reporting criteria, review published studies, and externally validate three well reported models using routinely collected health data from administrative claims and electronic health records. Methods We identified dementia prediction models that were developed between 2011 and 2020 and assessed if they could be externally validated given a set of model criteria. In addition, we externally validated three of these models (Walters’ Dementia Risk Score, Mehta’s RxDx-Dementia Risk Index, and Nori’s ADRD dementia prediction model) on a network of six observational health databases from the United States, United Kingdom, Germany and the Netherlands, including the original development databases of the models. Results We reviewed 59 dementia prediction models. All models reported the prediction method, development database, and target and outcome definitions. Less frequently reported by these 59 prediction models were predictor definitions (52 models) including the time window in which a predictor is assessed (21 models), predictor coefficients (20 models), and the time-at-risk (42 models). The validation of the model by Walters (development c-statistic: 0.84) showed moderate transportability (0.67–0.76 c-statistic). The Mehta model (development c-statistic: 0.81) transported well to some of the external databases (0.69–0.79 c-statistic). The Nori model (development AUROC: 0.69) transported well (0.62–0.68 AUROC) but performed modestly overall. Recalibration showed improvements for the Walters and Nori models, while recalibration could not be assessed for the Mehta model due to unreported baseline hazard. Conclusion We observed that reporting is mostly insufficient to fully externally validate published dementia prediction models, and therefore, it is uncertain how well these models would work in other clinical settings. We emphasize the importance of following established guidelines for reporting clinical prediction models. We recommend that reporting should be more explicit and have external validation in mind if the model is meant to be applied in different settings.

BackgroundThere is currently no consensus on the impact of class imbalance methods on the performance of clinical prediction models. We aimed to empirically investigate the impact of random oversampling and random undersampling, two commonly used class imbalance methods, on the internal and external validation performance of prediction models developed using observational health data.MethodsWe developed and externally validated prediction models for various outcomes of interest within a target population of people with pharmaceutically treated depression across four large observational health databases. We used three different classifiers (lasso logistic regression, random forest, XGBoost) and varied the target imbalance ratio. We evaluated the impact on model performance in terms of discrimination and calibration. Discrimination was assessed using the area under the receiver operating characteristic curve (AUROC) and calibration was assessed using calibration plots.ResultsWe developed and externally validated a total of 1,566 prediction models. On internal and external validation, random oversampling and random undersampling generally did not result in higher AUROCs. Moreover, we found overestimated risks, although this miscalibration could largely be corrected by recalibrating the models towards the imbalance ratios in the original dataset.ConclusionsOverall, we found that random oversampling or random undersampling generally does not improve the internal and external validation performance of prediction models developed in large observational health databases. Based on our findings, we do not recommend applying random oversampling or random undersampling when developing prediction models in large observational health databases.

Electronic health records (EHRs) consist of both structured data (eg, diagnostic codes) and unstructured data (eg, clinical notes). It is commonly believed that unstructured clinical narratives provide more comprehensive information. However, this assumption lacks large-scale validation and direct validation methods. This study aims to quantitatively compare the information in structured and unstructured EHR data and directly validate whether unstructured data offers more extensive information across a patient population. We analyzed both structured and unstructured data from patient records and visits in a large Dutch primary care EHR database between January 2021 and January 2024. Clinical concepts were identified from free-text notes using an extraction framework tailored for Dutch and compared with concepts from structured data. Concept embeddings were generated to measure semantic similarity between structured and extracted concepts through cosine similarity. A similarity threshold was systematically determined via annotated matches and minimized weighted Gini impurity. We then quantified the concept overlap between structured and unstructured data across various concept domains and patient populations. In a population of 1.8 million patients, only 13% of extracted concepts from patient records and 7% from individual visits had similar structured counterparts. Conversely, 42% of structured concepts in records and 25% in visits had similar matches in unstructured data. Condition concepts had the highest overlap, followed by measurements and drug concepts. Subpopulation visits, such as those with chronic conditions or psychological disorders, showed different proportions of data overlap, indicating varied reliance on structured versus unstructured data across clinical contexts. Our study demonstrates the feasibility of quantifying the information difference between structured and unstructured data, showing that the unstructured data provides important additional information in the studied database and populations. The annotated concept matches are made publicly available for the clinical natural language processing community. Despite some limitations, our proposed methodology proves versatile, and its application can lead to more robust and insightful observational clinical research.

Background Recent evidence suggests that there is often substantial variation in the benefits and harms across a trial population. We aimed to identify regression modeling approaches that assess heterogeneity of treatment effect within a randomized clinical trial. Methods We performed a literature review using a broad search strategy, complemented by suggestions of a technical expert panel. Results The approaches are classified into 3 categories: 1) Risk-based methods (11 papers) use only prognostic factors to define patient subgroups, relying on the mathematical dependency of the absolute risk difference on baseline risk; 2) Treatment effect modeling methods (9 papers) use both prognostic factors and treatment effect modifiers to explore characteristics that interact with the effects of therapy on a relative scale. These methods couple data-driven subgroup identification with approaches to prevent overfitting, such as penalization or use of separate data sets for subgroup identification and effect estimation. 3) Optimal treatment regime methods (12 papers) focus primarily on treatment effect modifiers to classify the trial population into those who benefit from treatment and those who do not. Finally, we also identified papers which describe model evaluation methods (4 papers). Conclusions Three classes of approaches were identified to assess heterogeneity of treatment effect. Methodological research, including both simulations and empirical evaluations, is required to compare the available methods in different settings and to derive well-informed guidance for their application in RCT analysis.

Background Baseline outcome risk can be an important determinant of absolute treatment benefit and has been used in guidelines for “personalizing” medical decisions. We compared easily applicable risk-based methods for optimal prediction of individualized treatment effects. Methods We simulated RCT data using diverse assumptions for the average treatment effect, a baseline prognostic index of risk, the shape of its interaction with treatment (none, linear, quadratic or non-monotonic), and the magnitude of treatment-related harms (none or constant independent of the prognostic index). We predicted absolute benefit using: models with a constant relative treatment effect; stratification in quarters of the prognostic index; models including a linear interaction of treatment with the prognostic index; models including an interaction of treatment with a restricted cubic spline transformation of the prognostic index; an adaptive approach using Akaike’s Information Criterion. We evaluated predictive performance using root mean squared error and measures of discrimination and calibration for benefit. Results The linear-interaction model displayed optimal or close-to-optimal performance across many simulation scenarios with moderate sample size ( N  = 4,250; ~ 785 events). The restricted cubic splines model was optimal for strong non-linear deviations from a constant treatment effect, particularly when sample size was larger ( N  = 17,000). The adaptive approach also required larger sample sizes. These findings were illustrated in the GUSTO-I trial. Conclusions An interaction between baseline risk and treatment assignment should be considered to improve treatment effect predictions.

Population-based studies can provide important evidence on the safety of COVID-19 vaccines. Using data from the United Kingdom, here we compare observed rates of thrombosis and thrombocytopenia following vaccination against SARS-CoV-2 and infection with SARS-CoV-2 with background (expected) rates in the general population. First and second dose cohorts for ChAdOx1 or BNT162b2 between 8 December 2020 and 2 May 2021 in the United Kingdom were identified. A further cohort consisted of people with no prior COVID-19 vaccination who were infected with SARS-Cov-2 identified by a first positive PCR test between 1 September 2020 and 2 May 2021. The fourth general population cohort for background rates included those people in the database as of 1 January 2017. In total, we included 3,768,517 ChAdOx1 and 1,832,841 BNT162b2 vaccinees, 401,691 people infected with SARS-CoV-2, and 9,414,403 people from the general population. An increased risk of venous thromboembolism was seen after first dose of ChAdOx1 (standardized incidence ratio: 1.12 [95% CI: 1.05 to 1.20]), BNT162b2 (1.12 [1.03 to 1.21]), and positive PCR test (7.27 [6.86 to 7.72]). Rates of cerebral venous sinus thrombosis were higher than otherwise expected after first dose of ChAdOx1 (4.14 [2.54 to 6.76]) and a SARS-CoV-2 PCR positive test (3.74 [1.56 to 8.98]). Rates of arterial thromboembolism after vaccination were no higher than expected but were increased after a SARS-CoV-2 PCR positive test (1.39 [1.21 to 1.61]). Rates of venous thromboembolism with thrombocytopenia were higher than expected after a SARS-CoV-2 PCR positive test (5.76 [3.19 to 10.40]). Population-based studies can provide information on the safety of COVID-19 vaccines. Here the authors report the rates thrombosis and thrombocytopenia after vaccination against and infection with SARS-CoV-2 in the United Kingdom and compare them with the background (expected) rates in the general population.

Real-world data (RWD) are increasingly used in health research and regulatory decision-making to assess the effectiveness, safety, and value of interventions in routine care. However, the heterogeneity of European health care systems, data capture methods, coding standards, and governance structures poses challenges for generating robust and reproducible real-world evidence. The European Health Data & Evidence Network (EHDEN) was established to address these challenges by building a large-scale federated data infrastructure that harmonizes RWD across Europe. This study aims to describe the composition and characteristics of the databases harmonized within EHDEN as of September 2024. We seek to provide transparency regarding the types of RWD available and their potential to support collaborative research and regulatory use. EHDEN recruited data partners through structured open calls. Selected data partners received funding and technical support to harmonize their data to the Observational Medical Outcomes Partnership Common Data Model (OMOP CDM), with assistance from certified small-to-medium enterprises trained through the EHDEN Academy. Each data source underwent an extract-transform-load process and data quality assessment using the data quality dashboard. Metadata-including country, care setting, capture method, and population criteria-were compiled in the publicly accessible EHDEN Portal. As of September 1, 2024, the EHDEN Portal includes 210 harmonized data sources from 30 countries. The highest representation comes from Italy (13%), Great Britain (12.5%), and Spain (11.5%). The mean number of persons per data source is 2,147,161, with a median of 457,664 individuals. Regarding care setting, 46.7% (n=98) of data sources reflect data exclusively from secondary care, 42.4% (n=89) from mixed care settings (both primary and secondary), and 11% (n=23) from primary care only. In terms of population inclusion criteria, 55.7% (n=117) of data sources include individuals based on health care encounters, 32.9% (n=69) through disease-specific data collection, and 11.4% (n=24) via population-based sources. Data capture methods also vary, with electronic health records (EHRs) being the most common. A total of 74.7% (n=157) of data sources use EHRs, and more than half of those (n=85) rely on EHRs as their sole method of data collection. Laboratory data are used in 29.5% (n=62) of data sources, although only one relies exclusively on laboratory data. Most laboratory-based data sources combine this method with other forms of data capture. EHDEN is the largest federated health data network in Europe, enabling standardized, General Data Protection Regulation-compliant analysis of RWD across diverse care settings and populations. This descriptive summary of the network's data sources enhances transparency and supports broader efforts to scale federated research. These findings demonstrate EHDEN's potential to enable collaborative studies and generate trusted evidence for public health and regulatory purposes.

A digital twin (DT), originally defined as a virtual representation of a physical asset, system, or process, is a new concept in health care. A DT in health care is not a single technology but a domain-adapted multimodal modeling approach incorporating the acquisition, management, analysis, prediction, and interpretation of data, aiming to improve medical decision-making. However, there are many challenges and barriers that must be overcome before a DT can be used in health care. In this viewpoint paper, we build on the current literature, address these challenges, and describe a dynamic DT in health care for optimizing individual patient health care journeys, specifically for women at risk for cardiovascular complications in the preconception and pregnancy periods and across the life course. We describe how we can commit multiple domains to developing this DT. With our cross-domain definition of the DT, we aim to define future goals, trade-offs, and methods that will guide the development of the dynamic DT and implementation strategies in health care.

Introduction: Real-world evidence (RWE) in health technology assessment (HTA) holds significant potential for informing healthcare decision-making. A multistakeholder workshop was organised by the European Health Data and Evidence Network (EHDEN) and the GetReal Institute to explore the status, challenges, and opportunities in incorporating RWE into HTA, with a focus on learning from regulatory initiatives such as the European Medicines Agency (EMA) Data Analysis and Real World Interrogation Network (DARWIN EU ® ). Methods: The workshop gathered key stakeholders from regulatory agencies, HTA organizations, academia, and industry for three panel discussions on RWE and HTA integration. Insights and recommendations were collected through panel discussions and audience polls. The workshop outcomes were reviewed by authors to identify key themes, challenges, and recommendations. Results: The workshop discussions revealed several important findings relating to the use of RWE in HTA. Compared with regulatory processes, its adoption in HTA to date has been slow. Barriers include limited trust in RWE, data quality concerns, and uncertainty about best practices. Facilitators include multidisciplinary training, educational initiatives, and stakeholder collaboration, which could be facilitated by initiatives like EHDEN and the GetReal Institute. Demonstrating the impact of “driver projects” could promote RWE adoption in HTA. Conclusion: To enhance the integration of RWE in HTA, it is crucial to address known barriers through comprehensive training, stakeholder collaboration, and impactful exemplar research projects. By upskilling users and beneficiaries of RWE and those that generate it, promoting collaboration, and conducting “driver projects,” can strengthen the HTA evidence base for more informed healthcare decisions.

Background A prediction model can be a useful tool to quantify the risk of a patient developing dementia in the next years and take risk-factor-targeted intervention. Numerous dementia prediction models have been developed, but few have been externally validated, likely limiting their clinical uptake. In our previous work, we had limited success in externally validating some of these existing models due to inadequate reporting. As a result, we are compelled to develop and externally validate novel models to predict dementia in the general population across a network of observational databases. We assess regularization methods to obtain parsimonious models that are of lower complexity and easier to implement. Methods Logistic regression models were developed across a network of five observational databases with electronic health records (EHRs) and claims data to predict 5-year dementia risk in persons aged 55–84. The regularization methods L1 and Broken Adaptive Ridge (BAR) as well as three candidate predictor sets to optimize prediction performance were assessed. The predictor sets include a baseline set using only age and sex, a full set including all available candidate predictors, and a phenotype set which includes a limited number of clinically relevant predictors. Results BAR can be used for variable selection, outperforming L1 when a parsimonious model is desired. Adding candidate predictors for disease diagnosis and drug exposure generally improves the performance of baseline models using only age and sex. While a model trained on German EHR data saw an increase in AUROC from 0.74 to 0.83 with additional predictors, a model trained on US EHR data showed only minimal improvement from 0.79 to 0.81 AUROC. Nevertheless, the latter model developed using BAR regularization on the clinically relevant predictor set was ultimately chosen as best performing model as it demonstrated more consistent external validation performance and improved calibration. Conclusions We developed and externally validated patient-level models to predict dementia. Our results show that although dementia prediction is highly driven by demographic age, adding predictors based on condition diagnoses and drug exposures further improves prediction performance. BAR regularization outperforms L1 regularization to yield the most parsimonious yet still well-performing prediction model for dementia.