Asset Details
Do you wish to reserve the book?
A framework for evaluating clinical artificial intelligence systems without ground-truth annotations
Hey, we have placed the reservation for you!
By the way, why not check out events that you can attend while you pick your title.
Oops! Something went wrong.
Looks like we were not able to place the reservation. Kindly try again later.
Are you sure you want to remove the book from the shelf?
A framework for evaluating clinical artificial intelligence systems without ground-truth annotations
Do you wish to request the book?
A framework for evaluating clinical artificial intelligence systems without ground-truth annotations
Please be aware that the book you have requested cannot be checked out. If you would like to checkout this book, you can reserve another copy
We have requested the book for you!
Your request is successful and it will be processed during the Library working hours. Please check the status of your request in My Requests.
Oops! Something went wrong.
Looks like we were not able to place your request. Kindly try again later.
Journal Article
A framework for evaluating clinical artificial intelligence systems without ground-truth annotations
2024
Overview
A clinical artificial intelligence (AI) system is often validated on data withheld during its development. This provides an estimate of its performance upon future deployment on data in the wild; those currently unseen but are expected to be encountered in a clinical setting. However, estimating performance on data in the wild is complicated by distribution shift between data in the wild and withheld data and the absence of ground-truth annotations. Here, we introduce SUDO, a framework for evaluating AI systems on data in the wild. Through experiments on AI systems developed for dermatology images, histopathology patches, and clinical notes, we show that SUDO can identify unreliable predictions, inform the selection of models, and allow for the previously out-of-reach assessment of algorithmic bias for data in the wild without ground-truth annotations. These capabilities can contribute to the deployment of trustworthy and ethical AI systems in medicine.
Estimating the performance of clinical AI systems on data in the wild is complicated by distribution shift and the absence of ground-truth annotations. Here, we introduce SUDO, a framework for more reliably evaluating AI systems on data in the wild.
Publisher
Nature Publishing Group UK,Nature Publishing Group,Nature Portfolio
