Journal Article

The Phases of Living Evidence Synthesis Using AI: Living Evidence Synthesis (Version 1)

Li, Ruixin, Luo, Xufei, Ma, Zhiming, Chen, Yaolong, Pu, Zhen, Yang, Kehu, Ge, Long, Lavis, John, Wang, Rui, Li, Caihong, Wang, Qi, Yang, Zhenzhen, Song, Xuping, Lian, Zhenjie, Feng, Lei

2026

Overview

Background:Living evidence (LE) synthesis refers to the method of continuously updating systematic evidence reviews to incorporate new evidence. It has emerged to address the limitations of the traditional systematic review process, particularly the absence of or delays in publication updates. The emergence of COVID-19 accelerated the progress in the field of LE synthesis, and currently, the applications of artificial intelligence (AI) in LE synthesis are expanding rapidly. However, in which phases of LE synthesis should AI be used remains an unanswered question.Objective:This study aims to (1) document the phases of LE synthesis where AI is used and (2) investigate whether AI improves the efficiency, accuracy, or utility of LE synthesis.Methods:We searched Web of Science, PubMed, the Cochrane Library, Epistemonikos, the Campbell Library, IEEE Xplore, medRxiv, COVID-19 Evidence Network to support Decision-making, and McMaster Health Forum. We used Covidence to facilitate the monthly screening and extraction processes to maintain the LE synthesis process. Studies that used or developed AI or semiautomated tools in the phases of LE synthesis were included.Results:A total of 24 studies were included, including 17 on LE syntheses, with 4 involving tool development, and 7 on living meta-analyses, with 3 involving tool development. First, a total of 34 AI or semiautomated tools were involved, comprising 12 AI tools and 22 semiautomated tools. The most frequently used AI or semiautomated tools were machine learning classifiers (n=5) and the Living Interactive Evidence synthesis platform (n=3). Second, 20 AI or semiautomated tools were used for the data extraction or collection and risk of bias assessment phase, and only 1 AI tool was used for the publication update phase. Third, 3 studies demonstrated the improvement in efficiency achieved based on time, workload, and conflict rate metrics. Nine studies applied AI or semiautomated tools in LE synthesis, obtaining a mean recall rate of 96.24%, and 6 studies achieved a mean F1-score of 92.17%. Additionally, 8 studies reported precision values ranging from 0.2% to 100%.Conclusions:AI and semiautomated tools primarily facilitate data extraction or collection and risk of bias assessment. The use of AI or semiautomated tools in LE synthesis improves efficiency, leading to high accuracy, recall, and F1-scores, while precision varies across tools.Trial Registration:OSF Registries 87tp4; https://osf.io/4fvdq/overview

• Publisher: Gunther Eysenbach MD MPH, Associate Professor,JMIR Publications Inc,JMIR Publications
• Subject: Accuracy / Artificial Intelligence / Bias / Clinical Information and Decision Making / Clinical trials / COVID-19 / Data collection / Decision making / Deep learning / Digital Health Meta-Research and Bibliographic Studies / Digital Health Reviews / Efficiency / Evidence-Based Practice and Medicine / Extraction / Health Services Research / Language / Large language models / Letters to the editor / Libraries / Machine learning / Meta-analysis / Natural language processing / Original Paper / Registration / Research Instruments, Questionnaires, and Tools / Risk assessment / Social networks / Systematic review