Journal Article

Agreement and utility of coded primary and secondary care data for long-term follow-up of clinical trial outcomes

Jones, Nicholas, Seeley, Anna E, Williams, Marney, de Lusignan, Simon, Hobbs, FD Richard, Sheppard, James P, McManus, Richard J, Wang, Ariel, Sydes, Matthew R

2025

Overview

Background Whilst interest in efficient trial design has grown with the use of electronic health records (EHRs) to collect trial outcomes, practical challenges remain. Commonly raised concerns often revolve around data availability, data quality and issues with data validation. This study aimed to assess the agreement between data collected on clinical trial participants from different sources to provide empirical evidence on the utility of EHRs for follow-up in randomised controlled trials (RCTs). Methods This retrospective, participant-level data utility comparison study was undertaken using data collected as part of a UK primary care-based, randomised controlled trial (OPTiMISE). The primary outcome measure was the recording of all-cause hospitalisation or mortality within 3 years post-randomisation and was assessed across (1) Coded primary care data; (2) Coded-plus-free-text primary care data; and (3) Coded secondary care and mortality data. Agreement levels across data sources were assessed using Fleiss’ Kappa (K). Kappa statistics were interpreted using an established framework, categorising agreement strength as follows: <0 (poor), 0.00–0.20 (slight), 0.21–0.40 (fair), 0.41–0.60 (moderate), 0.61–0.80 (substantial), and 0.81–1.00 (almost perfect) agreement. The impact of using different data sources to determine trial outcomes was assessed by replicating the trial’s original analyses. Results Almost perfect agreement was observed for mortality outcome across the three data sources (K = 0.94, 95%CI 0.91–0.98). Fair agreement (weak consistency) was observed for hospitalisation outcomes, including all-cause hospitalisation or mortality (K = 0.35, 95%CI 0.28–0.42), emergency hospitalisation (K = 0.39, 95%CI 0.33–0.46), and hospitalisation or mortality due to cardiovascular disease (K = 0.32, 95%CI 0.19–0.45). The overall trial results remained consistent across data sources for the primary outcome, albeit with varying precision. Conclusion Significant discrepancies according to data sources were observed in recording of secondary care outcomes. Investigators should be cautious when choosing which data source(s) to use to measure outcomes in trials. Future work on linking participant-level data across healthcare settings should consider the variations in diagnostic coding practices. Standardised definitions for outcome measures when using healthcare systems data and using data from different data sources for cross-checking and verification should be encouraged.

• Publisher: BioMed Central,BioMed Central Ltd,Springer Nature B.V,BMC
• Subject: Agreements / Antihypertensives / Blood pressure / Clinical trials / Data entry / Data linkage / Data utility / Electronic health records / Electronic health records (EHRs) / Electronic Health Records - statistics & numerical data / Electronic records / Evaluation / Evidence synthesis: from current practice to future advances / Female / Follow-Up Studies / Health Sciences / Healthcare systems data (HSD) / Hospitalization / Hospitalization - statistics & numerical data / Humans / Male / Medical records / Medicine / Medicine & Public Health / Methods / Mortality / Outcome and process assessment (Health Care) / Outcome Assessment, Health Care - methods / Outcome Assessment, Health Care - statistics & numerical data / Patients / Primary care / Primary Health Care - statistics & numerical data / Randomised controlled trial (RCT) / Randomized Controlled Trials as Topic - statistics & numerical data / Registration / Retrospective Studies / Routinely collected health data (RCHD) / Secondary Care - statistics & numerical data / Statistical Theory and Methods / Statistics for Life Sciences / Theory of Medicine/Bioethics / United Kingdom