Asset Details
Do you wish to reserve the book?
Automatic structuring of radiology reports with on-premise open-source large language models
Hey, we have placed the reservation for you!
By the way, why not check out events that you can attend while you pick your title.
Oops! Something went wrong.
Looks like we were not able to place the reservation. Kindly try again later.
Are you sure you want to remove the book from the shelf?
Automatic structuring of radiology reports with on-premise open-source large language models
Do you wish to request the book?
Automatic structuring of radiology reports with on-premise open-source large language models
Please be aware that the book you have requested cannot be checked out. If you would like to checkout this book, you can reserve another copy
We have requested the book for you!
Your request is successful and it will be processed during the Library working hours. Please check the status of your request in My Requests.
Oops! Something went wrong.
Looks like we were not able to place your request. Kindly try again later.
Journal Article
Automatic structuring of radiology reports with on-premise open-source large language models
2025
Overview
Objectives
Structured reporting enhances comparability, readability, and content detail. Large language models (LLMs) could convert free text into structured data without disrupting radiologists’ reporting workflow. This study evaluated an on-premise, privacy-preserving LLM for automatically structuring free-text radiology reports.
Materials and methods
We developed an approach to controlling the LLM output, ensuring the validity and completeness of structured reports produced by a locally hosted Llama-2-70B-chat model. A dataset with de-identified narrative chest radiograph (CXR) reports was compiled retrospectively. It included 202 English reports from a publicly available MIMIC-CXR dataset and 197 German reports from our university hospital. Senior radiologist prepared a detailed, fully structured reporting template with 48 question-answer pairs. All reports were independently structured by the LLM and two human readers. Bayesian inference (Markov chain Monte Carlo sampling) was used to estimate the distributions of Matthews correlation coefficient (MCC), with [−0.05, 0.05] as the region of practical equivalence (ROPE).
Results
The LLM generated valid structured reports in all cases, achieving an average MCC of 0.75 (94% HDI: 0.70–0.80) and F1 score of 0.70 (0.70–0.80) for English, and 0.66 (0.62–0.70) and 0.68 (0.64–0.72) for German reports, respectively. The MCC differences between LLM and humans were within ROPE for both languages: 0.01 (−0.05 to 0.07), 0.01 (−0.05 to 0.07) for English, and −0.01 (−0.07 to 0.05), 0.00 (−0.06 to 0.06) for German, indicating approximately comparable performance.
Conclusion
Locally hosted, open-source LLMs can automatically structure free-text radiology reports with approximately human accuracy. However, the understanding of semantics varied across languages and imaging findings.
Key Points
Question
Why has structured reporting not been widely adopted in radiology despite clear benefits and how can we improve this?
Findings
A locally hosted large language model successfully structured narrative reports, showing variation between languages and findings.
Critical relevance
Structured reporting provides many benefits, but its integration into the clinical routine is limited. Automating the extraction of structured information from radiology reports enables the capture of structured data while allowing the radiologist to maintain their reporting workflow.
Publisher
Springer Berlin Heidelberg,Springer Nature B.V
Subject
