Robust Transfer Principal Component Analysis with Rank Constraints
Principal component analysis (PCA), a well-established technique for data analysis and processing, provides a convenient form of dimensionality reduction that is effective for cleaning small Gaussian noises presented in the data. However, the applicability of standard principal component analysis in real scenarios is limited by its sensitivity to large errors. In this paper, we tackle the challenge problem of recovering data corrupted with errors of high magnitude by developing a novel robust transfer principal component analysis method. Our method is based on the assumption that useful information for the recovery of a corrupted data matrix can be gained from an uncorrupted related data matrix. Specifically, we formulate the data recovery problem as a joint robust principal component analysis problem on the two data matrices, with common principal components shared across matrices and individual principal components specific to each data matrix. The formulated optimization problem is a minimization problem over a convex objective function but with non-convex rank constraints. We develop an efficient proximal projected gradient descent algorithm to solve the proposed optimization problem with convergence guarantees. Our empirical results over image denoising tasks show the proposed method can effectively recover images with random large errors, and significantly outperform both standard PCA and robust PCA with rank constraints.
This paper considers robust principal component 
      analysis, from the approach of transfer learning. The goal is to obtain a 
      method that, according to the paper, can deal with not only small and/or 
      sparse errors, but also dense large errors, in the setting where there are 
      two data sources (two data matrices) which have some overlap in their 
      principal components. The authors then propose a rank-constrained 
      optimization problem that is the natural formulation, assuming sparse 
      errors; that is, they propose an objective which balances between the L2 
      loss in fitting the data matrix, plus an L1 penalty on the sparse 
      corruption. This, in principle, should allow the handling of sparse noise, 
      and also smaller dense noise. Instead of relaxing the rank constraints, 
      they propose a projected proximal type iterative method, where they 
      project back to matrices of appropriate rank, at every step. There 
      are several issues with this paper that if addressed, would significantly 
      strengthen the contribution. There are no conditions of success 
      that are discussed. That is, the authors do not describe, theoretically, 
      or even intuitively, what the conditions should be on the two data 
      matrices and a the noise corrupting them, that would allow their method to 
      succeed in finding the correct low dimensional structure. To be more 
      specific, if we compare to the papers that the authors cite in the 
      introduction, e.g., [11] or [14], or the numerous other papers that have 
      been written on various formulations of robust PCA, e.g., papers by E. 
      Candes et al., or papers by H. Xu et al., there, the authors give 
      algorithms along with specific conditions for which they provide 
      theoretical guarantees that their algorithms succeed. There does not seem 
      to be something similar here. Precisely because it is not the usual 
      setting, it would be very interesting to understand when transfer learning 
      might succeed, where other methods might fail. This is also very 
      important because some of the claims the authors make do not seem to be 
      backed up by a rigorous argument. The algorithm proposed is indeed 
      natural. However, it is solving a non-convex problem, because of the rank 
      constraints. By setting X_t or X_s to zero, it seems that one recovers the 
      low-rank plus sparse recovery problem (see, e.g., Chandrasekaran et al., 
      and Candes et al.). As mentioned previously, this is a well-studied and 
      fairly well understood problem. Does their algorithm have better 
      performance than the setting when the rank constraints are relaxed via the 
      nuclear norm? The wording in the paper seems to suggest this, but there is 
      nothing to support it.