Journal Article

A multimodal multi-scale transformer for virtual pretreatment patient-specific QA of SBRT using portal-dosimetry fluence maps

He, Xiao-Suo, You, Hong-Qiang, Zheng, Jia-Jun

2026

Overview

We developed a transformer-based multimodal neural network to predict the gamma passing rate (GPR) in stereotactic body radiation therapy (SBRT) patient-specific quality assurance. Using 1265 SBRT beams from two institutions, the model incorporated portal dose prediction fluence maps with beam complexity descriptors such as modulation complexity score and monitor units. A multi-scale visual-textual transformer, integrating a ViT encoder and feedforward network through a fusion head, was compared with state-of-the-art CNNs across nine gamma criteria. Our approach consistently achieved the lowest root mean squared error (RMSE) and mean absolute error (MAE), with values ranging from 0.785% to 4.258% and 0.418% to 3.197%, respectively, and ablation studies highlighted the necessity of multimodal fusion and multi-scale design. These results demonstrate superior predictive accuracy and generalizability, underscoring the potential of transformer-based multimodal learning to enhance treatment optimization and clinical QA efficiency.

• Publisher: Nature Publishing Group UK,Nature Publishing Group,Nature Portfolio
• Subject: 631/67 / 639/166 / 692/308 / 692/4028 / Datasets / Deep learning / Dosimetry / Efficiency / EPID / gamma passing rate / Humanities and Social Sciences / Humans / multi-scale ViT / multidisciplinary / multimodal learning / Neural networks / Neural Networks, Computer / Patient safety / Patients / Quality assurance / Quality Assurance, Health Care / Radiation therapy / Radiometry - methods / Radiosurgery - methods / Radiosurgery - standards / Radiotherapy Dosage / Radiotherapy Planning, Computer-Assisted - methods / SBRT / Science / Science (multidisciplinary) / Tumors