Noise-Tolerant Life-Long Matrix Completion via Adaptive Sampling
We study the problem of recovering an incomplete m × n matrix of rank r with columns arriving online over time. This is known as the problem of life-long matrix completion, and is widely applied to recommendation system, computer vision, system identification, etc. The challenge is to design provable algorithms tolerant to a large amount of noises, with small sample complexity. In this work, we give algorithms achieving strong guarantee under two realistic noise models. In bounded deterministic noise, an adversary can add any bounded yet unstructured noise to each column. For this problem, we present an algorithm that returns a matrix of a small error, with sample complexity almost as small as the best prior results in the noiseless case. For sparse random noise, where the corrupted columns are sparse and drawn randomly, we give an algorithm that exactly recovers an µ 0 -incoherent matrix by probability at least 1 − δ with sample complexity as small as O (µ 0 rn log(r/δ)). This result advances the state-of-the-art work and matches the lower bound in a worst case. We also study the scenario where the hidden matrix lies on a mixture of subspaces and show that the sample complexity can be even smaller. Our proposed algorithms perform well experimentally in both synthetic and real-world datasets.
This paper studies recovering an incomplete low rank matrix with noise, in an online fashion. The model is such that the learner are allowed to adaptively choose to unifomrly sample d features, or do a full measurement. Two noise models are studied, namely the bounded deterministic noise model, and the sparse random noise model. The authors provide recovery guarantees for both cases (robust recovery for the first case and exact recovery for the second case). The result matches or improved the state-of-art. However, notice that results in literature are for passive learning, where the authors assumes active learning/adaptive sampling, so the comparison is just illustrative.
This paper studies recovering an incomplete low rank matrix with noise, in an online fashion. The model is such that the learner are allowed to adaptively choose to unifomrly sample d features, or do a full measurement. Two noise models are studied, namely the bounded deterministic noise model, and the sparse random noise model. The authors provide recovery guarantees for both cases (robust recovery for the first case and exact recovery for the second case). The result matches or improved the state-of-art. However, notice that results in literature are for passive learning, where the authors assumes active learning/adaptive sampling, so the comparison is just illustrative.  This paper is well written and organized nicely. The contribution is in generally interesting. My main conerns are  (1) whether the adaptive sampling model used is significant/practical. The authors motivate this model via Netflix rating prediction where the company can sequentially send out questionnaries to each person for movies. While this is a valid rational if the adaptive sampling involves a moderately higher sampling rate, it does not justify the proposed algorithm where one may request a column being full observed (equivalent to ask a user of Netflix to watch all movies and rate!!) (2) The sparse random noise model, where the noise is assumed to be sparse and iid following a distribution. In literature, most sparse noise are modeled as arbitrary and adversarial, and hence the model studied by the authors are much easier. Even in such case, the tolerable noise is very low (indeed the cardinality of the support of the noise must be lower than the cardinality of the observed entries). It is not clear to me how much this can be improved. The lower bound presented does not cover this issue either.  Some minor comments: 1. subsubsection 3.1.2 should be a subsection, and 3.1.3, 3.1.4 combined should be a separate subsection too. 2. To avoid being misleading, the authors may want to mention that the lower bound does not rule out the possibility of improving the tolerance of the noise. 3. In theorem 6, the result depends on h very mildly, is there any intuition behind it?