Journal Article

Ten quick tips for effective dimensionality reduction

Nguyen, Lan Huong, Holmes, Susan

2019

Overview

Both a means of denoising and simplification, it can be beneficial for the majority of modern biological datasets, in which it’s not uncommon to have hundreds or even millions of simultaneous measurements collected for a single sample. Because of “the curse of dimensionality,” many statistical methods lack power when applied to high-dimensional data. Formally, the Marchenko–Pastur distribution asymptotically models the distribution of the singular values of large random matrices. [...]for datasets large in both the number of observations and features, you use a rule of retaining only eigenvalues outside the support of the fitted Marchenko–Pastur distribution; however, remember that this applies only when your data have at least thousands of samples and thousands of features. [...]the height-to-width ratio of a PCA plot should be consistent with the ratio between the corresponding eigenvalues. Because eigenvalues reflect the variance in coordinates of the associated PCs, you only need to ensure that in the plots, one \"unit\" in direction of one PC has the same length as one \"unit\" in direction of another PC. Because batch effects can confound the signal of interest, it is a good practice to check for their presence and, if found, to remove them before proceeding with further downstream analysis.

• Publisher: Public Library of Science,Public Library of Science (PLoS)
• Subject: Artificial intelligence / Bioinformatics / Biological research / Biology and Life Sciences / Computational Biology - education / Computational Biology - methods / Computer and Information Sciences / Data analysis / Data compression / Data reduction / Data visualization / Databases, Factual / Datasets / Dimensional analysis / Education / Eigenvalues / Gene expression / Humans / Linear algebra / Machine Learning / Medicine and Health Sciences / Methods / Models, Statistical / Noise reduction / Physical Sciences / Research and Analysis Methods / Signal Processing, Computer-Assisted / Statistical methods / Variables