Paper

Towards a Single Unified Model for Effective Detection, Segmentation, and Diagnosis of Eight Major Cancers Using a Large Collection of CT Scans

Yao, Jiawen, Zhou, Jian, Lu, Le, Yan, Ke, Fang, Wei, Xia, Yingda, Yin, Xiaoli, Zhang, Jianpeng, Zhou, Jingren, Wang, Qifeng, Ye, Xianghua, Chen, Jieneng, Yuille, Alan, Zhang, Ling, Yu, Qihang, Yuan, Mingze, Wang, Fakai, Qiu, Mingyan, Tang, Yuxing, Zhao, Yuqian, Liu, Zaiyi, Chen, Xin, Zhou, Bo, Xu, Minfeng, Shi, Yu

2023

Overview

Human readers or radiologists routinely perform full-body multi-organ multi-disease detection and diagnosis in clinical practice, while most medical AI systems are built to focus on single organs with a narrow list of a few diseases. This might severely limit AI's clinical adoption. A certain number of AI models need to be assembled non-trivially to match the diagnostic process of a human reading a CT scan. In this paper, we construct a Unified Tumor Transformer (UniT) model to detect (tumor existence and location) and diagnose (tumor characteristics) eight major cancer-prevalent organs in CT scans. UniT is a query-based Mask Transformer model with the output of multi-organ and multi-tumor semantic segmentation. We decouple the object queries into organ queries, detection queries and diagnosis queries, and further establish hierarchical relationships among the three groups. This clinically-inspired architecture effectively assists inter- and intra-organ representation learning of tumors and facilitates the resolution of these complex, anatomically related multi-organ cancer image reading tasks. UniT is trained end-to-end using a curated large-scale CT images of 10,042 patients including eight major types of cancers and occurring non-cancer tumors (all are pathology-confirmed with 3D tumor masks annotated by radiologists). On the test set of 631 patients, UniT has demonstrated strong performance under a set of clinically relevant evaluation metrics, substantially outperforming both multi-organ segmentation methods and an assembly of eight single-organ expert models in tumor detection, segmentation, and diagnosis. Such a unified multi-cancer image reading model (UniT) can significantly reduce the number of false positives produced by combined multi-system models. This moves one step closer towards a universal high-performance cancer screening tool.

• Publisher: Cornell University Library, arXiv.org
• Subject: Cancer / Computed tomography / Diagnosis / Image segmentation / Medical imaging / Medical screening / Organs / Queries / Task complexity / Transformers / Tumors