Identifying anomalous samples from highly complex and unstructured data is a crucial but challenging task in a variety of intelligent systems. In this paper, we present a novel deep anomaly detection framework named AnoDM (standing for Anomaly detection based on unsupervised Disentangled representation learning and Manifold learning). The disentanglement learning is currently implemented by β-VAE for automatically discovering interpretable factorized latent representations in a completely unsupervised manner. The manifold learning is realized by t-SNE for projecting the latent representations to a 2D map. We define a new anomaly score function by combining β-VAE's reconstruction error in the raw feature space and local density estimation in the t-SNE space. AnoDM was evaluated on both image and time-series data and achieved better results than models that use just one of the two measures and other deep learning methods.
This paper presents a novel deep anomaly detection model. It combines two existing models: B-VAE and t-SNE. The B-VAE is trained unsupervised and learns an encoder and decoder which provide both an embedding and a reconstruction. Using t-SNE to reduce its dimensionality, the embedding is projected into a 2 dimensional space. An anomaly score function is defined that combines the reconstruction error and the distance in t-SNE space to the K nearest neighbor(s). Experiments are conducted with several image datasets (MNIST,FMNIST,CIFAR10,SmallNORB) and one timeseries dataset (Arrhythmia). For the image sets, the B-VAE model is implemented with a CNN, while for timeseries, a TCN is used. Comparisons are conducted showing the approach to be beat other SOT unsupervised methods, AnoGAN and ADGAN, by 63% and 22% respectively for MNIST and 8% and 2% for FMNIST (in terms of error reduction). For CIFAR-10 and FMNIST it is even demonstrated to beat a supervised SOT method CapsNET. Another experiment shows that t_SNE dramatically improves the performance over B-VAE alone. For the timeseries, the approach is not compared to other SOT approaches as the authors only provide an experiment showing that TCN beats CNN and LSTM for the implementation of the B-VAE. In addition the authors study the effect of the various parameters of the system, in particular the effect of the B in B-VAE and of alpha, the mixing factor between reconstruction error and kNN distance in t_SNE. 3D plots give a good idea on how to select optimal values for the various datasets. The impact of B is also shown on the t-SNE map for MNIST. Finally an ablation studies compares on MNIST the performance of the approach with t-SNE alone, reconstruction alone, and latent distance. On average over 4 digits taken as anomaly, the proposed approach dramatically outperforms the others.
PROS:
* The proposed approach improves over the SOT of competitive recent methods for anomaly detection on four image datasets.
* The authors make an effort to abstract the approach into a framework where other deep learning models and dimensionality reduction techniques can be used. They illustrate this by using a TCN instead of a CNN for the timeseries example.
* The parameter studies and ablation studies are informative and answer many of the questions i had as i read the paper.
* The paper is relatively clearly written (at least sufficiently to easily understand the technical details).
CONS:
* The novelty of the paper is limited as it is mostly a combination of 2 existing methods.
* The timeseries dataset is not compared to SOT methods (although the authors claim SOT in the conclusion).
* A pseudo-code algorithm is not provided, making it unlikely someone can reproduce the method.
* There are many typos an grammatical errors
* The paper could have been shortened. 10 pages is too long.
Overall, because of the good performance and thoughtful ablation studies, and despite the limited novelty, I think the paper makes a good contribution to anomaly detection.