Collaborative Filtering with Graph Information: Consistency and Scalable Methods
Low rank matrix completion plays a fundamental role in collaborative filtering applications, the key idea being that the variables lie in a smaller subspace than the ambient space. Often, additional information about the variables is known, and it is reasonable to assume that incorporating this information will lead to better predictions. We tackle the problem of matrix completion when pairwise relationships among variables are known, via a graph. We formulate and derive a highly efficient, conjugate gradient based alternating minimization scheme that solves optimizations with over 55 million observations up to 2 orders of magnitude faster than state-of-the-art (stochastic) gradient-descent based methods. On the theoretical front, we show that such methods generalize weighted nuclear norm formulations, and derive statistical consistency guarantees. We validate our results on both real and synthetic datasets.
Paper 1260 tackles the problem of matrix completion with side information represented as a graph. The authors claim that this is a general framework in that the side information can come from many different kinds of sources. Major contributions of this paper are scalable learning algorithm for matrix completition with graph structures, connecting it to existing works in matrix factorization literature, and verifying the proposed method theoretically as well as experimentally.As the authors claim, this problem is widely applicable such as recommendation systems with social network information. Considering the size of such dataset, it is important to bulid a scalable algorithm as the authors proposed.This paper is well-structured and well-written. It is good to have both theoretical analysis of consistency and experiments with real data. Also, it is useful to connect the proposed method in the context of wider convex optimization and noisy matrix approximation.Regarding the experiment (Section 6): some baselines in Table 1 are too simple; no one might be interested in comparison with Global/User/Movie mean. I suggest comparing against more recent, state-of-the-art methods such asSalakhutdinov and Mnih, Probabilistic Matrix Factorization [NIPS 2008], Mackey et al., Divide-and-Conquer Matrix Factorization [NIPS 2011], Lee et al., Local Low-Rank Matrix Approximation [ICML 2013].It is impressive that the proposed method runs in O(10^3) seconds with dataset in Table 2. However, to claim scalability, I recommend running the algorithm on larger dataset, such as Netflix or Yahoo Music. This way it is also possible comparing RMSE score with other methods as well.Minor comments: 1) Why Figure 1 and Figure 2 have different format of x-axis? I recommend using format of Figure 2 (10^1, 10^2, ...) instead of plotting log(time) directly. Log-scale without specifying base is ambiguous. 2) Citations are not in NIPS format.