Explore the vast range of titles available.

We develop an automatic method for synaptic partner identification in insect brains and use it to predict synaptic partners in a whole-brain electron microscopy dataset of the fruit fly. The predictions can be used to infer a connectivity graph with high accuracy, thus allowing fast identification of neural pathways. To facilitate circuit reconstruction using our results, we develop CIRCUITMAP, a user interface add-on for the circuit annotation tool CATMAID.A deep-learning-based approach enables automatic identification of synaptically connected neurons in electron microscopy datasets of the fly brain.

We developed an advanced deep learning approach called local shape descriptors (LSDs) to enable analysis of large electron microscopy datasets with increased efficiency. This technique will speed processing of future petabyte-sized datasets and democratize connectomics research by enabling these analyses using modest computational infrastructure available to most laboratories.

We present an auxiliary learning task for the problem of neuron segmentation in electron microscopy volumes. The auxiliary task consists of the prediction of local shape descriptors (LSDs), which we combine with conventional voxel-wise direct neighbor affinities for neuron boundary detection. The shape descriptors capture local statistics about the neuron to be segmented, such as diameter, elongation, and direction. On a study comparing several existing methods across various specimen, imaging techniques, and resolutions, auxiliary learning of LSDs consistently increases segmentation accuracy of affinity-based methods over a range of metrics. Furthermore, the addition of LSDs promotes affinity-based segmentation methods to be on par with the current state of the art for neuron segmentation (flood-filling networks), while being two orders of magnitudes more efficient—a critical requirement for the processing of future petabyte-sized datasets. During segmentation of neurons in electron microscopy datasets, auxiliary learning via the prediction of local shape descriptors increases efficiency, which is important for the processing of datasets of ever-increasing size.

The cerebellum is thought to help detect and correct errors between intended and executed commands 1 , 2 and is critical for social behaviours, cognition and emotion 3 – 6 . Computations for motor control must be performed quickly to correct errors in real time and should be sensitive to small differences between patterns for fine error correction while being resilient to noise 7 . Influential theories of cerebellar information processing have largely assumed random network connectivity, which increases the encoding capacity of the network’s first layer 8 – 13 . However, maximizing encoding capacity reduces the resilience to noise 7 . To understand how neuronal circuits address this fundamental trade-off, we mapped the feedforward connectivity in the mouse cerebellar cortex using automated large-scale transmission electron microscopy and convolutional neural network-based image segmentation. We found that both the input and output layers of the circuit exhibit redundant and selective connectivity motifs, which contrast with prevailing models. Numerical simulations suggest that these redundant, non-random connectivity motifs increase the resilience to noise at a negligible cost to the overall encoding capacity. This work reveals how neuronal network structure can support a trade-off between encoding capacity and redundancy, unveiling principles of biological network architecture with implications for the design of artificial neural networks. Mapping of the mouse cerebellar cortex using 3D reconstruction from electron microscopy, as well as numerical simulation of neuronal activity, shows non-random redundancy of connectivity that may favour resilient learning over encoding capacity.

The development of high-resolution microscopes has made it possible to investigate cellular processes in 3D and over time. However, observing fast cellular dynamics remains challenging because of photobleaching and phototoxicity. Here we report the implementation of two content-aware frame interpolation (CAFI) deep learning networks, Zooming SlowMo and Depth-Aware Video Frame Interpolation, that are highly suited for accurately predicting images in between image pairs, therefore improving the temporal resolution of image series post-acquisition. We show that CAFI is capable of understanding the motion context of biological structures and can perform better than standard interpolation methods. We benchmark CAFI’s performance on 12 different datasets, obtained from four different microscopy modalities, and demonstrate its capabilities for single-particle tracking and nuclear segmentation. CAFI potentially allows for reduced light exposure and phototoxicity on the sample for improved long-term live-cell imaging. The models and the training and testing data are available via the ZeroCostDL4Mic platform. Content-aware frame interpolation (CAFI) improves the temporal resolution in time-lapse imaging by accurately predicting images in between image pairs. By allowing fewer frames to be imaged, CAFI also enables gentler live-cell imaging.

Through digital imaging, microscopy has evolved from primarily being a means for visual observation of life at the micro- and nano-scale, to a quantitative tool with ever-increasing resolution and throughput. Artificial intelligence, deep neural networks, and machine learning are all niche terms describing computational methods that have gained a pivotal role in microscopy-based research over the past decade. This Roadmap encompasses key aspects of how machine learning is applied to microscopy image data, with the aim of gaining scientific knowledge by improved image quality, automated detection, segmentation, classification and tracking of objects, and efficient merging of information from multiple imaging modalities. We aim to give the reader an overview of the key developments and an understanding of possibilities and limitations of machine learning for microscopy. It will be of interest to a wide cross-disciplinary audience in the physical sciences and life sciences.

Producing dense 3D reconstructions from biological imaging data is a challenging instance segmentation task that requires significant ground-truth training data for effective and accurate deep learning-based models. Generating training data requires intense human effort to annotate each instance of an object across serial section images. Our focus is on the especially complicated brain neuropil, comprising an extensive interdigitation of dendritic, axonal, and glial processes visualized through serial section electron microscopy. We developed a novel deep learning-based method to generate dense 3D segmentations rapidly from sparse 2D annotations of a few objects on single sections. Models trained on the rapidly generated segmentations achieved similar accuracy as those trained on expert dense ground-truth annotations. Human time to generate annotations was reduced by three orders of magnitude and could be produced by non-expert annotators. This capability will democratize generation of training data for large image volumes needed to achieve brain circuits and measures of circuit strengths.

Neural representations of head direction have been discovered in many species. A large body of theoretical work has proposed that the dynamics associated with these representations is generated, maintained, and updated by recurrent network structures called ring attractors. We performed electron microscopy-based circuit reconstruction and RNA profiling of identified cell types in the heading direction system of Drosophila melanogaster to directly determine the underlying neural network. We identified network motifs that have been hypothesized to maintain the heading representation in darkness, update it when the animal turns, and tether it to visual cues. Functional studies provided additional support for the proposed roles of individual circuit elements. We also discovered recurrent connections between neuronal arbors with mixed pre- and post-synaptic specializations. Overall, our results confirm that the Drosophila heading direction network contains the core components of a ring attractor while also revealing unpredicted structural features that might enhance the network's computational power. Footnotes * This revision includes additional data and analysis, as well as improvements to several figures and text.

Mapping nanoscale neuronal morphology with molecular annotations is critical for understanding healthy and dysfunctional brain circuits. Current methods are constrained by image segmentation errors and by sample defects (e.g., signal gaps, section loss). Genetic strategies promise to overcome these challenges by using easily distinguishable cell identity labels. However, multicolor approaches are spectrally limited in diversity, whereas nucleic acid barcoding lacks a cell-filling morphology signal for segmentation. Here, we introduce PRISM (Protein-barcode Reconstruction via Iterative Staining with Molecular annotations), a platform that integrates combinatorial delivery of anti-genically distinct, cell-filling proteins with tissue expansion, multi-cycle imaging, barcode-augmented reconstruction, and molecular annotation. Protein barcodes increase label diversity by >750-fold over multicolor labeling and enable morphology reconstruction with intrinsic error correction. We acquired a ~10 million μm volume of mouse hippocampal area CA2/3, multiplexed across 23 barcode antigen and synaptic marker channels. By combining barcodes with shape information, we achieve an 8x increase in automatic tracing accuracy of genetically labelled neurons. We demonstrate PRISM supports automatic proofreading across micron-scale spatial gaps and reconnects neurites across discontinuities spanning hundreds of microns. Using PRISM's molecular annotation capability, we map the distribution of synapses onto traced neural morphology, characterizing challenging synaptic structures such as thorny excrescences (TEs), and discovering a size correlation among spatially proximal TEs on the same dendrite. PRISM thus supports self-correcting neuron reconstruction with molecular context.