Journal Article

Comparative evaluation of YOLOv8, YOLOv11, and RT-DETR for automated microcarrier colonization assessment in bioreactor cell cultures

Menta Hernández, Ramón, Lombardo, Eleuterio, Avivar‐Valderas, Alvaro, Menéndez, Beatriz, Blanco Sánchez, Cristina, Canales, Natalia, Costa, Marta H. G., Mancheño-Corvo, Pablo, Serra, Margarida, García Domínguez, Gonzalo, Fernández de Sevilla, Manuel T., Valero, Raúl

2026

Overview

IntroductionMonitoring microcarrier colonization during mesenchymal stromal cell (MSC) expansion in bioreactors is essential for process control. Current manual quantification methods are time-consuming, subjective, and prone to inter-operator variability. This study presents a proof-of-concept evaluation of deep learning-based object detection for automated microcarrier colonization quantification in bioreactor cell culturesMethodsThree state-of-the-art architectures—YOLOv8, YOLOv11, and RT-DETR—were compared within a unified experimental framework. A dataset of 699 fluorescence microscopy images containing 46,982 annotated microcarriers (classified as colonized or non-colonized) was used for training and evaluation. Model selection followed a three-stage Successive Halving Algorithm (SHA) strategy to efficiently identify the best-performing architecture across 12 configurations without requiring high-end computational infrastructure. COCO-pretrained transfer learning was applied across all models.ResultsYOLOv8-l emerged as the top-performing model, achieving mAP50–95 of 0.855 on validation and 0.787 on test, with a colonization estimation error (Mean Absolute Error) of 11.75%. The dataset size proved sufficient for CNN-based architectures, while also revealing the higher data demands of transformer-based detectors.DiscussionThese results demonstrate that automated deep learning pipelines can reliably quantify microcarrier colonization, offering a practical alternative to manual assessment in cell therapy manufacturing. The findings also suggest that newer YOLO models do not necessarily yield improvements in domain-specific biomedical tasks

• Publisher: Frontiers Media SA,Frontiers Media S.A
• Subject: Accuracy / Automation / Bioreactors / Business metrics / Cell culture / Cell therapy / Colonization / Computer vision / Concept learning / Datasets / Deep learning / Fluorescence microscopy / Good Manufacturing Practice / Manufacturing / Mesenchymal stem cells / mesenchymal stromal cell / Microscopy / Transfer learning / YOLO