Many applications require recovering a ground truth low-rank matrix from noisy observations of the entries. In practice, this is typically formulated as weighted low-rank approximation problem and solved using non-convex optimization heuristics such as alternating minimization. Such non-convex techniques have little guarantees. Even worse, weighted low-rank approximation is NP-hard for even the most simple case when the ground truth is a rank-1 matrix. #R##N#In this paper, we provide provable recovery guarantee in polynomial time for a natural class of matrices and weights. In particular, we bound the spectral norm of the difference between the recovered matrix and the ground truth, by the spectral norm of the weighted noise plus an additive error term that decreases exponentially with the number of rounds of alternating minimization. This provides the first theoretical result for weighted low-rank approximation via alternating minimization with non-binary deterministic weights. It is a significant generalization of the results for matrix completion, the special case with binary weights, since our assumptions are similar or weaker than those made in existing works. #R##N#The key technical challenge is that under non-binary deterministic weights, naive alternating minimization steps will destroy the incoherency and spectral properties of the intermediate solution, which are needed for making progress towards the ground truth. One of our key technical contributions is a whitening step that maintains these properties of the intermediate solution after each round, which may be applied to alternating minimization for other problems and thus is of independent interest.
['Yuanzhi Li', 'Yingyu Liang', 'Andrej Risteski']
Recovery guarantee of weighted low-rank approximation via alternating minimization