Efficient Anomaly Detection via Matrix Sketching
We consider the problem of finding anomalies in high-dimensional data using popular PCA based anomaly scores. The naive algorithms for computing these scores explicitly compute the PCA of the covariance matrix which uses space quadratic in the dimensionality of the data. We give the first streaming algorithms that use space that is linear or sublinear in the dimension. We prove general results showing that any sketch of a matrix that satisfies a certain operator norm guarantee can be used to approximate these scores. We instantiate these results with powerful matrix sketching techniques such as Frequent Directions and random projections to derive efficient and practical algorithms for these problems, which we validate over real-world data sets. Our main technical contribution is to prove matrix perturbation inequalities for operators arising in the computation of these measures.
Authors develop two new algorithms to find anomalies in high dimensional data when the row data is streaming in. Those algorithms are both two-pass algorithms to approximately compute the rank k leverage score as well as rank k projection distance with running complexity time of O(nd\ell), where n and d are the size of an input matrix A and \ell is independent of n and d, depends only on rank k and some other parameters. While the first algorithm requires O(d\ell) memory, the second one uses only O(log(d)) memory. However, the cost of achieving such reduction is losing accuracy as the first algorithm posses pointwise guarantees from row space approximation and the second one holds average-case guarantees from columns space approximation. The approximation rates are derived analytically and illustrated numerically.   In general, the paper is well-written, well-referenced throughout, and in appropriate format/structure. The practical algorithms presented in this paper contributes to the existing literature by proposing two simple methods which do work for any sketch matrix satisfying certain inequalities ((1) and (2)). The paper can be considered for publication in NIPS 2018 Proceedings if the authors can address the following minor comments in the manuscript:  1. Line 145: Replace "other parameters" with "k and \epsilon".  2. Line 184: Replace "2" with "(2)".  3. Line 187: Remove "for" before "e.g.".  4. Line 189: Replace "a n\times n" with "an n\times n". 5. Line 191: Replace "a \ell\times \ell" with "an \ell\times \ell".   6. Line 198: There is a typo in "psedorandomness".  7. Line 230: To keep the statement of Algorithm 1 consistent with the numerical results, replace "Frequent Directions" in Input of this algorithm with "Random Column Projections". Accordingly, lines 156-157 as well as Footnote 2 should be modified.  8. Figures 2-4: In all six plots, replace "Column Projection" and "Row Projection" with "Algorithm 1" and "Algorithm 2", respectively.   At the end end, I have a minor question: To avoid degeneracy in the spectrum of matrix A in theorems 1 and 3, should we assume that that A is (k,\delta) and (k-1,\delta)-separated?