Speedup Matrix Completion with Side Information: Application to Multi-Label Learning
In standard matrix completion theory, it is required to have at least O(n ln 2 n) observed entries to perfectly recover a low-rank matrix M of size n × n, leading to a large number of observations when n is large. In many real tasks, side information in addition to the observed entries is often available. In this work, we develop a novel theory of matrix completion that explicitly explore the side information to reduce the requirement on the number of observed entries. We show that, under appropriate conditions, with the assistance of side information matrices, the number of observed entries needed for a perfect recovery of matrix M can be dramatically reduced to O(ln n). We demonstrate the effectiveness of the proposed approach for matrix completion in transductive incomplete multi-label learning.
The paper studies the matrix completion problem in the 
      context of collaborative filtering or multi-label learning. The 
      contribution over existing work is that the authors account for side 
      information on the rows and columns of the matrix in their model and show 
      how exploiting side information can bring about improvements in sample 
      complexity over the standard matrix completion formulation. In particular 
      the authors study a model for matrix completion where the matrix of 
      interest M can be decomposed as M = A Z_0 B^T with A and B is the known 
      side information . When A and B are low dimensional this formulation has 
      fewer degrees of freedom since the unknown matrix Z_0 is much smaller than 
      M itself which intuitively explains why one should expect better sample 
      complexity guarantees. The authors analyze a very natural 
      extension of the usual nuclear norm minimization program. They show that 
      the sample complexity required to guarantee unique minimizer is O(\mu^2 r 
      (r_a + r_b) \log n) for a n \times n matrix of rank r with Z_0 of 
      dimension r_a \times r_b and where \mu is related to the usual incoherence 
      parameter used in the literature. This implies that if the side 
      information is low dimensional (r_a, r_b small) the sample complexity is 
      sublinear in the matrix dimensions, which is an improvement over standard 
      results. The authors complement the theoretical results with 
      simulations and experiments in the context of multi-label learning. The 
      simulations show that their algorithm is both computationally and 
      statistically more efficient that singular value thresholding, one of the 
      standard matrix completion algorithms. Their algorithm also outperforms a 
      number of baselines on several data sets in the multi-label learning 
      application. The paper studies an interesting problem and 
      contributes to the body of work on matrix completion by capturing side 
      information. The results are interesting and the paper is well written. 
      The proof techniques are novel but not particularly innovative given the 
      work on matrix completion. In light of this, I think the contribution is 
      somewhat small. In particular, it would be nice to know the necessary 
      conditions in the noiseless case and the guarantees for the noisy version 
      of the matrix completion problem. The experiments are fairly convincing 
      but could be presented better. Graphs in lieu of tables would lead to 
      improved readability while also saving space. Questions: 1. In 
      the main paper there is no discussion about the role of \Omega_0 and 
      \Omega_1 or any discussion about the condition \Omega_1 \ge q \Omega_0. It 
      would be nice to provide some intuition as to why such a condition 
      necessary and regimes under which it is satisfied. 2. Lemma 4 in the 
      appendix requires that \Omega_0 \le |\Omega| \le \Omega_1, but in the main 
      theorem there is no upper bound on the sample complexity. This seems like 
      an error, but I'm more concerned with why there is an upper bound on 
      |\Omega| in the first place. Is this condition necessary? What is the 
      intuition for it? 3. Why are there test instances held out of the 
      experiments? Why not just randomly sample the label matrix and test on the 
      unobserved entries of that matrix? 4. The paper title is misleading -- 
      Speedup implies computationally, which was not the focus of the paper. I 
      read the paper thinking it would be much more of an algorithmic paper than 
      it was.