Journal Article

International evaluation of an AI system for breast cancer screening

Gilbert, Fiona J., Melnick, David, Ledsam, Joseph R., Mostofi, Hormuz, McKinney, Scott Mayer, Corrado, Greg S., Darzi, Ara, Tse, Daniel, Young, Kenneth C., Back, Trevor, Reicher, Joshua Jay, Garcia-Vicente, Florencia, Suleyman, Mustafa, Halling-Brown, Mark, Hassabis, Demis, Sieniek, Marcin, Karthikesalingam, Alan, Godbole, Varun, Shetty, Shravya, Kelly, Christopher J., Sidebottom, Richard, De Fauw, Jeffrey, Etemadi, Mozziyar, King, Dominic, Romera-Paredes, Bernardino, Ashrafian, Hutan, Chesus, Mary, Godwin, Jonathan, Jansen, Sunny, Antropova, Natasha, Peng, Lily

2020

Overview

Screening mammography aims to identify breast cancer at earlier stages of the disease, when treatment can be more successful 1 . Despite the existence of screening programmes worldwide, the interpretation of mammograms is affected by high rates of false positives and false negatives 2 . Here we present an artificial intelligence (AI) system that is capable of surpassing human experts in breast cancer prediction. To assess its performance in the clinical setting, we curated a large representative dataset from the UK and a large enriched dataset from the USA. We show an absolute reduction of 5.7% and 1.2% (USA and UK) in false positives and 9.4% and 2.7% in false negatives. We provide evidence of the ability of the system to generalize from the UK to the USA. In an independent study of six radiologists, the AI system outperformed all of the human readers: the area under the receiver operating characteristic curve (AUC-ROC) for the AI system was greater than the AUC-ROC for the average radiologist by an absolute margin of 11.5%. We ran a simulation in which the AI system participated in the double-reading process that is used in the UK, and found that the AI system maintained non-inferior performance and reduced the workload of the second reader by 88%. This robust assessment of the AI system paves the way for clinical trials to improve the accuracy and efficiency of breast cancer screening. An artificial intelligence (AI) system performs as well as or better than radiologists at detecting breast cancer from mammograms, and using a combination of AI and human inputs could help to improve screening efficiency.

• Publisher: Nature Publishing Group UK,Nature Publishing Group
• Subject: 631/67/1347 / 692/308/2778 / Analysis / Artificial intelligence / Artificial Intelligence - standards / Breast cancer / Breast Neoplasms - diagnostic imaging / Cancer / Cancer screening / Clinical medicine / Clinical trials / Datasets / Deep learning / Diagnosis / Early Detection of Cancer - methods / Early Detection of Cancer - standards / Evaluation / Female / Human performance / Humanities and Social Sciences / Humans / Mammography / Mammography - standards / Medical screening / multidisciplinary / Performance evaluation / Readers / Reproducibility of Results / Science / Science (multidisciplinary) / Software / Technology application / United Kingdom / United States / Womens health / Workload