Provable Non-convex Robust PCA
We propose a new method for robust PCA -the task of recovering a low-rank matrix from sparse corruptions that are of unknown value and support. Our method involves alternating between projecting appropriate residuals onto the set of lowrank matrices, and the set of sparse matrices; each projection is non-convex but easy to compute. In spite of this non-convexity, we establish exact recovery of the low-rank matrix, under the same conditions that are required by existing methods (which are based on convex optimization). For an m×n input matrix (m ≤ n), our method has a running time of O r 2 mn per iteration, and needs O (log(1/ )) iterations to reach an accuracy of . This is close to the running times of simple PCA via the power method, which requires O (rmn) per iteration, and O (log(1/ )) iterations. In contrast, the existing methods for robust PCA, which are based on convex optimization, have O m 2 n complexity per iteration, and take O (1/ ) iterations, i.e., exponentially more iterations for the same accuracy.
Experiments on both synthetic and real data establishes the improved speed and accuracy of our method over existing convex implementations.
This paper is about developing non-convex counter part to Robust PCA. Robust PCA is a popular convex optimization scheme for decomposing a low rank matrix which is corrupted with sparse perturbations into its sparse and low rank components. However, the current methods based on convex optimization are computationally prohibitive as they require either matrix inversion and or full singular value decomposition. This paper develops a non-convex approach to this problem. The authors' simulations indicate that this non-convex algorithm can be much faster. The authors also provide some theoretical analysis for their algorithm.I think the subject of this paper is very timely and well suited for NIPS. Overall I find the paper to be interesting and well written. The technical issues I had raised have been resolved (for the most part) in the authors' response. There are still some issues with authors' claim of matching convex schemes (which I've explained below). However, I trust the authors to address this issue in the final version of the paper. Regardless of this issue I think this paper is a must accept at NIPS and I have updated my score. There also a couple of minor mistakes in the authors' response which I detail below.- Modified Pros:-- well written-- suited for NIPS-- new algorithm-- study of non-convex problems (very few such studies in the literature)-- real data experiments- Modified Cons:-- proven in an easier regime compared to convex counter parts-- need knowledge of problem parameters such as rank and incoherenceDetailed comments regarding the authors' response:1) ``We match KHZ 11"The authors do not match the deterministic results of HKZ11. They match it for the case where one is interested in sparsity per row/column. The interesting part about the deterministic result of HKZ is that their conditions can not only give this result but also nearly match the results of CLMW11 for the random models.2) ``Comparison with CLMW11:"Even with these added assumptions the author can not at this point match the results of CLMW11. My point is that I think the authors should be careful in their comparison and claims. That being said, I completely agree with the authors that these results can be matched with significantly more evolved proofs.