Journal Article

ROSIE: AI generation of multiplex immunofluorescence staining from histopathology images

Behera, Vivek, Wu, Eric, Schürch, Christian M., Li, Li, Zou, James, Charville, Gregory W., Bieniosek, Matthew, Raman, Arjun, Huyghe, Jeroen R., Wu, Zhenqin, Makky, Ahmad, Trevino, Alexandro E., Giba, Hannah, Thakkar, Nitya, Peters, Ulrike, Li, Christopher I., Mayer, Aaron T.

2025

Overview

Hematoxylin and eosin (H&E) is a common and inexpensive histopathology assay. Though widely used and information-rich, it cannot directly inform about specific molecular markers, which require additional experiments to assess. To address this gap, we present ROSIE, a deep-learning framework that computationally imputes the expression and localization of dozens of proteins from H&E images. Our model is trained on a dataset of over 1300 paired and aligned H&E and multiplex immunofluorescence (mIF) samples from over a dozen tissues and disease conditions, spanning over 16 million cells. Validation of our in silico mIF staining method on held-out H&E samples demonstrates that the predicted biomarkers are effective in identifying cell phenotypes, particularly distinguishing lymphocytes such as B cells and T cells, which are not readily discernible with H&E staining alone. Additionally, ROSIE facilitates the robust identification of stromal and epithelial microenvironments and immune cell subtypes like tumor-infiltrating lymphocytes (TILs), which are important for understanding tumor-immune interactions and can help inform treatment strategies in cancer research. Hematoxylin and eosin (H&E) staining is a widely used method in histopathology, but it cannot directly inform about specific molecular markers. Here, the authors present ROSIE, a deep-learning framework that computationally imputes the expression and localisation of dozens of proteins from H&E images.

• Publisher: Nature Publishing Group UK,Nature Publishing Group,Nature Portfolio
• Subject: 13/1 / 14/63 / 631/114/1305 / 631/1647/2067 / 631/67/2321 / Biomarkers / Biomarkers, Tumor - metabolism / Datasets / Deep Learning / Eosine Yellowish-(YS) - chemistry / Fluorescent Antibody Technique - methods / Hematoxylin - chemistry / Histopathology / Humanities and Social Sciences / Humans / Image Processing, Computer-Assisted - methods / Immune system / Immunofluorescence / Localization / Lymphocytes / Lymphocytes B / Lymphocytes T / Lymphocytes, Tumor-Infiltrating - immunology / Lymphocytes, Tumor-Infiltrating - pathology / Medical imaging / Methods / Microenvironments / multidisciplinary / Multiplexing / Neoplasms - pathology / Phenotypes / Protein expression / Proteins / Science / Science (multidisciplinary) / Staining / Staining and Labeling - methods / Stains & staining / Tumor Microenvironment / Tumor-infiltrating lymphocytes / Tumors