Journal Article

Evaluation and development of deep neural networks for image super-resolution in optical microscopy

Liu, Chong, Li, Dong, Jiang, Tao, Li, Di, Guo Yuting, Dai Qionghai, Chang, Qiao

2021

Overview

Deep neural networks have enabled astonishing transformations from low-resolution (LR) to super-resolved images. However, whether, and under what imaging conditions, such deep-learning models outperform super-resolution (SR) microscopy is poorly explored. Here, using multimodality structured illumination microscopy (SIM), we first provide an extensive dataset of LR–SR image pairs and evaluate the deep-learning SR models in terms of structural complexity, signal-to-noise ratio and upscaling factor. Second, we devise the deep Fourier channel attention network (DFCAN), which leverages the frequency content difference across distinct features to learn precise hierarchical representations of high-frequency information about diverse biological structures. Third, we show that DFCAN’s Fourier domain focalization enables robust reconstruction of SIM images under low signal-to-noise ratio conditions. We demonstrate that DFCAN achieves comparable image quality to SIM over a tenfold longer duration in multicolor live-cell imaging experiments, which reveal the detailed structures of mitochondrial cristae and nucleoids and the interaction dynamics of organelles and cytoskeleton.This study explores the performance of deep-learning models for super-resolution imaging and introduces models that utilize frequency content information in the Fourier domain to improve SIM reconstruction under low-SNR conditions.

• Publisher: Nature Publishing Group
• Subject: Artificial neural networks / Cristae / Cytoskeleton / Deep learning / Domains / Image quality / Image reconstruction / Image resolution / Information processing / Light microscopy / Machine learning / Microscopy / Mitochondria / Neural networks / Nucleoids / Optical microscopy / Organelles / Signal to noise ratio