Explore the vast range of titles available.

Recently, multi-modal vision-language foundation models have gained significant attention in the medical field. While these models offer great opportunities, they still face crucial challenges, such as the requirement for fine-grained knowledge understanding in computer-aided diagnosis and the capability of utilizing very limited or even no task-specific labeled data in real-world clinical applications. In this study, we present MaCo, a masked contrastive chest X-ray foundation model that tackles these challenges. MaCo explores masked contrastive learning to simultaneously achieve fine-grained image understanding and zero-shot learning for a variety of medical imaging tasks. It designs a correlation weighting mechanism to adjust the correlation between masked chest X-ray image patches and their corresponding reports, thereby enhancing the model’s representation learning capabilities. To evaluate the performance of MaCo, we conducted extensive experiments using 6 well-known open-source X-ray datasets. The experimental results demonstrate the superiority of MaCo over 10 state-of-the-art approaches across tasks such as classification, segmentation, detection, and phrase grounding. These findings highlight the significant potential of MaCo in advancing a wide range of medical image analysis tasks. Multi-modal foundation models are increasingly important in medical applications. Here, authors show a masked contrastive chest X-ray model that achieves fine-grained image understanding and zero-shot capabilities, outperforming existing methods

Cancer is a complex disease composing systemic alterations in multiple scales. In this study, we develop the Tumor Multi-Omics pre-trained Network (TMO-Net) that integrates multi-omics pan-cancer datasets for model pre-training, facilitating cross-omics interactions and enabling joint representation learning and incomplete omics inference. This model enhances multi-omics sample representation and empowers various downstream oncology tasks with incomplete multi-omics datasets. By employing interpretable learning, we characterize the contributions of distinct omics features to clinical outcomes. The TMO-Net model serves as a versatile framework for cross-modal multi-omics learning in oncology, paving the way for tumor omics-specific foundation models.

The success of training computer-vision models heavily relies on the support of large-scale, real-world images with annotations. Yet such an annotation-ready dataset is difficult to curate in pathology due to the privacy protection and excessive annotation burden. To aid in computational pathology, synthetic data generation, curation, and annotation present a cost-effective means to quickly enable data diversity that is required to boost model performance at different stages. In this study, we introduce a large-scale synthetic pathological image dataset paired with the annotation for nuclei semantic segmentation, termed as Synthetic Nuclei and annOtation Wizard (SNOW). The proposed SNOW is developed via a standardized workflow by applying the off-the-shelf image generator and nuclei annotator. The dataset contains overall 20k image tiles and 1,448,522 annotated nuclei with the CC-BY license. We show that SNOW can be used in both supervised and semi-supervised training scenarios. Extensive results suggest that synthetic-data-trained models are competitive under a variety of model training settings, expanding the scope of better using synthetic images for enhancing downstream data-driven clinical tasks.

Breast carcinoma is the most common cancer among women worldwide that consists of a heterogeneous group of subtype diseases. The whole-slide images (WSIs) can capture the cell-level heterogeneity, and are routinely used for cancer diagnosis by pathologists. However, key driver genetic mutations related to targeted therapies are identified by genomic analysis like high-throughput molecular profiling. In this study, we develop a deep-learning model to predict the genetic mutations and biological pathway activities directly from WSIs. Our study offers unique insights into WSI visual interactions between mutation and its related pathway, enabling a head-to-head comparison to reinforce our major findings. Using the histopathology images from the Genomic Data Commons Database, our model can predict the point mutations of six important genes (AUC 0.68–0.85) and copy number alteration of another six genes (AUC 0.69–0.79). Additionally, the trained models can predict the activities of three out of ten canonical pathways (AUC 0.65–0.79). Next, we visualized the weight maps of tumor tiles in WSI to understand the decision-making process of deep-learning models via a self-attention mechanism. We further validated our models on liver and lung cancers that are related to metastatic breast cancer. Our results provide insights into the association between pathological image features, molecular outcomes, and targeted therapies for breast cancer patients.

Overcoming barriers on the use of multi-center data for medical analytics is challenging due to privacy protection and data heterogeneity in the healthcare system. In this study, we propose the Distributed Synthetic Learning (DSL) architecture to learn across multiple medical centers and ensure the protection of sensitive personal information. DSL enables the building of a homogeneous dataset with entirely synthetic medical images via a form of GAN-based synthetic learning. The proposed DSL architecture has the following key functionalities: multi-modality learning, missing modality completion learning, and continual learning. We systematically evaluate the performance of DSL on different medical applications using cardiac computed tomography angiography (CTA), brain tumor MRI, and histopathology nuclei datasets. Extensive experiments demonstrate the superior performance of DSL as a high-quality synthetic medical image provider by the use of an ideal synthetic quality metric called Dist-FID. We show that DSL can be adapted to heterogeneous data and remarkably outperforms the real misaligned modalities segmentation model by 55% and the temporal datasets segmentation model by 8%. Here the authors present Distributed Synthetic Learning, a system that addresses data privacy, isolated data islands, and heterogeneity concerns in healthcare analytics by learning to generate state-of-the-art synthetic data for downstream tasks.

The efficacy of large language models (LLMs) in domain-specific medicine, particularly for managing complex diseases such as osteoarthritis (OA), remains largely unexplored. This study focused on evaluating and enhancing the clinical capabilities and explainability of LLMs in specific domains, using OA management as a case study. A domain-specific benchmark framework was developed to evaluate LLMs across a spectrum from domain-specific knowledge to clinical applications in real-world clinical scenarios. DocOA, a specialized LLM designed for OA management integrating retrieval-augmented generation and instructional prompts, was developed. It can identify the clinical evidence upon which its answers are based through retrieval-augmented generation, thereby demonstrating the explainability of those answers. The study compared the performance of GPT-3.5, GPT-4, and a specialized assistant, DocOA, using objective and human evaluations. This study introduces a novel benchmark framework that assesses the domain-specific abilities of LLMs in multiple aspects, highlights the limitations of generalized LLMs in clinical contexts, and demonstrates the potential of tailored approaches for developing domain-specific medical LLMs.

Foundation models, often pre-trained with large-scale data, have achieved paramount success in jump-starting various vision and language applications. Recent advances further enable adapting foundation models in downstream tasks efficiently using only a few training samples, e.g., in-context learning. Yet, the application of such learning paradigms in medical image analysis remains scarce due to the shortage of publicly accessible data and benchmarks. In this paper, we aim at approaches adapting the foundation models for medical image classification and present a novel dataset and benchmark for the evaluation, i.e., examining the overall performance of accommodating the large-scale foundation models downstream on a set of diverse real-world clinical tasks. We collect five sets of medical imaging data from multiple institutes targeting a variety of real-world clinical tasks (22,349 images in total), i.e., thoracic diseases screening in X-rays, pathological lesion tissue screening, lesion detection in endoscopy images, neonatal jaundice evaluation, and diabetic retinopathy grading. Results of multiple baseline methods are demonstrated using the proposed dataset from both accuracy and cost-effective perspectives.

PurposeTo build and validate a magnetic resonance imaging-based radiomics model to preoperatively evaluate tumor budding (TB) in locally advanced rectal cancer (LARC).MethodsPathologically confirmed LARC cases submitted to preoperative rectal MRI in two distinct hospitals were enrolled in this retrospective study and assigned to cohort 1 (training set, n = 77; test set, n = 51) and cohort 2 (validation set, n = 96). Radiomics features were obtained from multiple sequences, comprising high-resolution T2, contrast-enhanced T1, and diffusion-weighted imaging (T2WI, CE-T1WI, and DWI, respectively). The least absolute shrinkage and selection operator (LASSO) was utilized to select the optimal features from T2WI, CE-T1WI, DWI, and the combination of multi-sequences, respectively. A support vector machine (SVM) classifier was utilized to construct various radiomics models for discriminating the TB grades. Receiver operating characteristic curve analysis and decision curve analysis (DCA) were carried out to determine the diagnostic value.ResultsFive optimal features associated with TB grade were determined from combined multi-sequence data. Accordingly, a radiomics model based on combined multi-sequences had an area under the curve of 0.796, with an accuracy of 81.2% in the validation set, showing a better performance in comparison with other models in both cohorts (p < 0.05). DCA exhibited a clinical benefit for this radiomics model.ConclusionThe novel MRI-based radiomics model combining multiple sequences is an effective and non-invasive approach for evaluating TB grade preoperatively in patients with LARC.

Background KIT is frequently mutated in gastrointestinal stromal tumors (GISTs), and the treatment of GISTs largely relies on targeting KIT currently. In this study, we aimed to investigate the role of sprouty RTK signaling antagonist 4 (SPRY4) in GISTs and related mechanisms. Methods Ba/F3 cells and GIST-T1 cell were used as cell models, and mice carrying germline KIT/V558A mutation were used as animal model. Gene expression was examined by qRT-PCR and western blot. Protein association was examined by immunoprecipitation. Results Our study revealed that KIT increased the expression of SPRY4 in GISTs. SPRY4 was found to bind to both wild-type KIT and primary KIT mutants in GISTs, and inhibited KIT expression and activation, leading to decreased cell survival and proliferation mediated by KIT. We also observed that inhibition of SPRY4 expression in KIT V558A/WT mice led to increased tumorigenesis of GISTs in vivo. Moreover, our results demonstrated that SPRY4 enhanced the inhibitory effect of imatinib on the activation of primary KIT mutants, as well as on cell proliferation and survival mediated by the primary KIT mutants. However, in contrast to this, SPRY4 did not affect the expression and activation of drug-resistant secondary KIT mutants, nor did it affect the sensitivity of secondary KIT mutants to imatinib. These findings suggested that secondary KIT mutants regulate a different downstream signaling cascade than primary KIT mutants . Conclusions Our results suggested that SPRY4 acts as negative feedback of primary KIT mutants in GISTs by inhibiting KIT expression and activation. It can increase the sensitivity of primary KIT mutants to imatinib. In contrast, secondary KIT mutants are resistant to the inhibition of SPRY4.

Purpose Disease-free survival (DFS) after neoadjuvant chemoradiotherapy (nCRT) is an important factor in affecting the quality of life and determining the subsequent treatment procedures for patients with locally advanced rectal cancer (LARC). This study aimed to develop a novel prognostic model for predicting the DFS in patients with LARC following nCRT and to verify its effectiveness. Materials and methods Patients with LARC who underwent magnetic resonance imaging (MRI) and nCRT at our institution between November 2017 and March 2022 were enrolled in this retrospective study. Clinicopathologic data and MRI indicators of all patients were collected and evaluated. All patients were divided into DFS and non-DFS groups according to the presence or absence of local recurrence or distant metastasis. The differences in the b -value threshold ( b threshold ) and apparent diffusion coefficient (ADC) values between the DFS and non-DFS groups were compared. The Cox analyses were used to determine the risk factors in predicting the DFS. A merged model was established based on the risk factors, and a nomogram was constructed. The predictive performances of the merged model were validated using the receiver-operating characteristic (ROC) and decision curve analysis (DCA). Results Of the 524 patients enrolled, 132 who underwent surgical resection post-nCRT were included in the final analysis. The post-neoadjuvant therapy pathological N stage, extramural venous invasion (EMVI), and b threshold were independent factors in predicting the DFS. The C-index of the model was 0.688. The area under the curve (AUC) of the nomogram in predicting the 1-, 3-, and 5-year survival rates of patients was 0.731, 0.723, and 0.779, respectively. The DCA demonstrated that the merged model had a greater advantage than either the “all” or the “none” scheme when the threshold probability was between 0.1 and 0.65. Conclusion A merged model based on the b threshold value and clinicopathological features showed the potential to predict the prognosis of patients with LARC after nCRT and surgery.