Journal Article

MRIQC: Advancing the automatic prediction of image quality in MRI from unseen sites

Birman, Daniel, Schaer, Marie, Esteban, Oscar, Koyejo, Oluwasanmi O., Poldrack, Russell A., Gorgolewski, Krzysztof J.

2017

Overview

Quality control of MRI is essential for excluding problematic acquisitions and avoiding bias in subsequent image processing and analysis. Visual inspection is subjective and impractical for large scale datasets. Although automated quality assessments have been demonstrated on single-site datasets, it is unclear that solutions can generalize to unseen data acquired at new sites. Here, we introduce the MRI Quality Control tool (MRIQC), a tool for extracting quality measures and fitting a binary (accept/exclude) classifier. Our tool can be run both locally and as a free online service via the OpenNeuro.org portal. The classifier is trained on a publicly available, multi-site dataset (17 sites, N = 1102). We perform model selection evaluating different normalization and feature exclusion approaches aimed at maximizing across-site generalization and estimate an accuracy of 76%±13% on new sites, using leave-one-site-out cross-validation. We confirm that result on a held-out dataset (2 sites, N = 265) also obtaining a 76% accuracy. Even though the performance of the trained classifier is statistically above chance, we show that it is susceptible to site effects and unable to account for artifacts specific to new sites. MRIQC performs with high accuracy in intra-site prediction, but performance on unseen sites leaves space for improvement which might require more labeled data and new approaches to the between-site variability. Overcoming these limitations is crucial for a more objective quality assessment of neuroimaging data, and to enable the analysis of extremely large and multi-site samples.

• Publisher: Public Library of Science,Public Library of Science (PLoS)
• Subject: Artificial intelligence / Autism / Automation / Biology and Life Sciences / Brain - diagnostic imaging / Brain research / Classifiers / Computer and Information Sciences / Computer science / Data acquisition / Datasets / Engineering and Technology / Humans / Image Enhancement - methods / Image Interpretation, Computer-Assisted - methods / Image processing / Image quality / Information processing / Inspection / Magnetic resonance imaging / Magnetic Resonance Imaging - methods / Medical imaging / Medicine and Health Sciences / Methods / Neuroimaging / Neuroimaging - methods / Neurology / NMR / Noise / Nuclear magnetic resonance / Observer Variation / Quality assessment / Quality control / Research and Analysis Methods / Social Sciences / Software / Statistical analysis / Statistical methods / Visual perception