Journal Article

A flexible framework for anomaly Detection via dimensionality reduction

Kunz, M., Bassett, Bruce A., Vafaei Sadr, Alireza

2023

Overview

Anomaly detection is challenging, especially for large datasets in high dimensions. Here, we explore a general anomaly detection framework based on dimensionality reduction and unsupervised clustering. DRAMA is released as a general python package that implements the general framework with a wide range of built-in options. This approach identifies the primary prototypes in the data with anomalies detected by their large distances from the prototypes, either in the latent space or in the original, high-dimensional space. DRAMA is tested on a wide variety of simulated and real datasets, in up to 3000 dimensions, and is found to be robust and highly competitive with commonly used anomaly detection algorithms, especially in high dimensions. The flexibility of the DRAMA framework allows for significant optimization once some examples of anomalies are available, making it ideal for online anomaly detection, active learning, and highly unbalanced datasets. Besides, DRAMA naturally provides clustering of outliers for subsequent analysis.

• Publisher: Springer London,Springer Nature B.V
• Subject: Algorithms / Anomalies / Artificial Intelligence / Clustering / Computational Biology/Bioinformatics / Computational Science and Engineering / Computer Science / Data analysis / Data Mining and Knowledge Discovery / Datasets / Image Processing and Computer Vision / Optimization / Outliers (statistics) / Probability and Statistics in Computer Science / Prototypes / Reduction / S.I. : 2019 India Intl. Congress on Computational Intelligence