Journal Article

Intra-database validation of case-identifying algorithms using reconstituted electronic health records from healthcare claims data

Abouelfath, Abdelilah, Diez, Pauline, Gross-Goupil, Marine, Roumiguié, Mathieu, Lassalle, Régis, Le Moulec, Sylvestre, Moore, Nicholas, Lignot, Séverine, Maillart, Elisabeth, Bosco-Levy, Pauline, Soulié, Michel, Thurin, Nicolas H., Blin, Patrick, Lamarque, Stéphanie, Bignon, Emmanuelle, Droz-Perroteau, Cécile, Rouyer, Magali, Debouverie, Marc, Louapre, Céline, Guillemin, Francis, Jové, Jérémy, Brochet, Bruno, Heinzlef, Olivier

2021

Overview

Background Diagnosis performances of case-identifying algorithms developed in healthcare database are usually assessed by comparing identified cases with an external data source. When this is not feasible, intra-database validation can present an appropriate alternative. Objectives To illustrate through two practical examples how to perform intra-database validations of case-identifying algorithms using reconstituted Electronic Health Records (rEHRs). Methods Patients with 1) multiple sclerosis (MS) relapses and 2) metastatic castration-resistant prostate cancer (mCRPC) were identified in the French nationwide healthcare database (SNDS) using two case-identifying algorithms. A validation study was then conducted to estimate diagnostic performances of these algorithms through the calculation of their positive predictive value (PPV) and negative predictive value (NPV). To that end, anonymized rEHRs were generated based on the overall information captured in the SNDS over time (e.g. procedure, hospital stays, drug dispensing, medical visits) for a random selection of patients identified as cases or non-cases according to the predefined algorithms. For each disease, an independent validation committee reviewed the rEHRs of 100 cases and 100 non-cases in order to adjudicate on the status of the selected patients (true case/ true non-case), blinded with respect to the result of the corresponding algorithm. Results Algorithm for relapses identification in MS showed a 95% PPV and 100% NPV. Algorithm for mCRPC identification showed a 97% PPV and 99% NPV. Conclusion The use of rEHRs to conduct an intra-database validation appears to be a valuable tool to estimate the performances of a case-identifying algorithm and assess its validity, in the absence of alternative.

• Publisher: BioMed Central,BioMed Central Ltd,Springer Nature B.V,BMC
• Subject: Algorithms / Analysis / Cancer therapies / Care and treatment / Case-identifying algorithm / Claims database / Codes / Data analysis / Databases, Factual / Delivery of Health Care / Diagnosis / Drugs / Electronic Health Records / Electronic records / Health Sciences / Hospitals / Humans / Laboratories / Life Sciences / Male / Management / Medical records / Medicine / Medicine & Public Health / Metastasis / Multiple sclerosis / Neoplasm Recurrence, Local / Patients / Performance evaluation / Prognosis / Prostate Cancer / Reconstituted electronic health record / Research Article / Statistical Theory and Methods / statistics and modelling / Statistics for Life Sciences / Theory of Medicine/Bioethics / Validation studies / Validation study