Journal Article

Retrospective batch analysis to evaluate the diagnostic accuracy of a clinically deployed AI algorithm for the detection of acute pulmonary embolism on CTPA

Hoesein, Firdaus A. Mohamed, Veldhuis, Wouter B, de Jong, Pim A, Dekker, Lisette, van den Hoven, Andor F, Langius-Wiffen, Eline, Nijholt, Ingrid M, Boomsma, Martijn F

2023

Overview

PurposeTo generate and extend the evidence on the clinical validity of an artificial intelligence (AI) algorithm to detect acute pulmonary embolism (PE) on CT pulmonary angiography (CTPA) of patients suspected of PE and to evaluate the possibility of reducing the risk of missed findings in clinical practice with AI-assisted reporting.MethodsConsecutive CTPA scan data of 3316 patients referred because of suspected PE between 24-2-2018 and 31-12-2020 were retrospectively analysed by a CE-certified and FDA-approved AI algorithm. The output of the AI was compared with the attending radiologists’ report. To define the reference standard, discordant findings were independently evaluated by two readers. In case of disagreement, an experienced cardiothoracic radiologist adjudicated.ResultsAccording to the reference standard, PE was present in 717 patients (21.6%). PE was missed by the AI in 23 patients, while the attending radiologist missed 60 PE. The AI detected 2 false positives and the attending radiologist 9. The sensitivity for the detection of PE by the AI algorithm was significantly higher compared to the radiology report (96.8% vs. 91.6%, p < 0.001). Specificity of the AI was also significantly higher (99.9% vs. 99.7%, p = 0.035). NPV and PPV of the AI were also significantly higher than the radiology report.ConclusionThe AI algorithm showed a significantly higher diagnostic accuracy for the detection of PE on CTPA compared to the report of the attending radiologist. This finding indicates that missed positive findings could be prevented with the implementation of AI-assisted reporting in daily clinical practice.Critical relevance statementMissed positive findings on CTPA of patients suspected of pulmonary embolism can be prevented with the implementation of AI-assisted care.Key pointsThe AI algorithm showed excellent diagnostic accuracy detecting PE on CTPA.Accuracy of the AI was significantly higher compared to the attending radiologist.Highest diagnostic accuracy can likely be achieved by radiologists supported by AI.Our results indicate that implementation of AI-assisted reporting could reduce the number of missed positive findings.

• Publisher: Springer Nature B.V
• Subject: Accuracy / Algorithms / Angiography / Artificial intelligence / Clinical medicine / Diagnostic systems / Embolisms / Medical diagnosis / Pulmonary embolisms / Radiology