A Pseudo-Bayesian Algorithm for Robust PCA
Commonly used in many applications, robust PCA represents an algorithmic attempt to reduce the sensitivity of classical PCA to outliers. The basic idea is to learn a decomposition of some data matrix of interest into low rank and sparse components, the latter representing unwanted outliers. Although the resulting problem is typically NP-hard, convex relaxations provide a computationally-expedient alternative with theoretical support. However, in practical regimes performance guarantees break down and a variety of non-convex alternatives, including Bayesian-inspired models, have been proposed to boost estimation quality. Unfortunately though, without additional a priori knowledge none of these methods can significantly expand the critical operational range such that exact principal subspace recovery is possible. Into this mix we propose a novel pseudo-Bayesian algorithm that explicitly compensates for design weaknesses in many existing non-convex approaches leading to state-of-the-art performance with a sound analytical foundation.
The paper presents a variational algorithm for robust Bayesian PCA.
Update after author feedback: I thank the authors for the clarification of the purpose of the paper. Including that in the text might make it easier for all your readers to appreciate the paper.  ---------------------------------------  Technical quality:       The method seems reasonable. The experiments are weak: all the experiments appear at least partly synthetic in terms of added noise.         Novelty:       The method is an interesting novel combination of existing ideas. The main novelty appears to be a new bound on a joint column-wise and row-wise prior.        Impact:       On paper, the method looks very good and one would expect it to attract a lot of users. I am however underwhelmed by the experiments: the authors have do not seem to have been able to find a real practical killer application for their method which raises some doubt of how useful it will be in practice.        Clarity:       The paper is difficult to read because there appears to be too much material crammed in. There are many comments that could be expanded or supported by relevant references (e.g. lines 34, 82-83, Eq. (7) and its relation to ARD, line 135, proof of Proposition 1, ...). I feel the submission might work better as a journal paper than a brief conference paper.        I am not sure why the authors have chosen to call their method pseudo-Bayesian, which seems like a very uninformative name. The method seems like a good examples of a variational algorithm in the spirit of Jordan et al., i.e. not of the KL divergence minimising type. At least I would find the term "variational" much more informative as a characterisation.              The experiments are described very briefly and many important parameters and their values are not described, especially how r is chosen. The metrics used in all the experiments should be presented and the parameters defining success justified. The sensitivity of the results to any arbitrary parameters should be analysed.              There is no discussion on weaknesses and limitations of the method.