Unified View of Matrix Completion under General Structural Constraints
Matrix completion problems have been widely studied under special low dimensional structures such as low rank or structure induced by decomposable norms. In this paper, we present a unified analysis of matrix completion under general low-dimensional structural constraints induced by any norm regularization. We consider two estimators for the general problem of structured matrix completion, and provide unified upper bounds on the sample complexity and the estimation error. Our analysis relies on generic chaining, and we establish two intermediate results of independent interest: (a) in characterizing the size or complexity of low dimensional subsets in high dimensional ambient space, a certain partial complexity measure encountered in the analysis of matrix completion problems is characterized in terms of a well understood complexity measure of Gaussian widths, and (b) it is shown that a form of restricted strong convexity holds for matrix completion problems under general norm regularization. Further, we provide several non-trivial examples of structures included in our framework, notably including the recently proposed spectral k-support norm.
This paper introduces a general analysis of matrix completion with norm regularization. The analysis setting can be thought of as a generalization of the setting of Negahban and Wainwright, in which we assume that the target matrix has small L-\infty norm, and attempt to prove that natural optimization formulations stably recover the target matrix, up to an error which is proportional to the maximum of the noise level and a floor (which is a consequence of only assuming the target is not spiky). The paper motivates this problem through several examples, including spectral-k-support norms, and norms for cut matrices. It states results showing that the estimation error and number of samples required is controlled by the squared Gaussian width of the descent cone of the regularizer at the target, times a logarithmic oversampling factor. This has a nice interpretation - namely, that "in general" the number of measurements in the standard basis only needs to be a logarithmic factor larger than the number of Gaussian measurements. While it is not clear how mainstream some of the examples are, it is certainly welcome to have general tools which show that accurate recovery from elementwise subsamples is not a phenomenon which is restricted to the nuclear norm.  Smaller issuesThe main theorem becomes vacuous when the number of sampled entries approaches w_G(E_R)^4. This seems to mostly be a technical issue, but it should be corrected. For general norms w_G could be quite small, and taking w_G^4 samples could represent an interesting scenario. There are imprecisions in the mathematics. The calculations in the supplement seem to move back and forth between a Bernoulli sampling model (which is implicitly used in A.1.1, (18), (a) - otherwise || PO[ X ] ||_F^2 is not a sum of independent random variables, due to collisions) and the uniform sampling model, which is used elsewhere. Proposition 4: \alpha_sp( \hat{\Delta} ) \in A(c_0, E_R ). Should just be \hat{\Delta} \in A(c_0, E_R). While the paper analyzes matrix completion in any norm (and hence with any prior information that can be encoded in a norm), there are some norms that are not compatible with the spikiness condition. For example, the paper gives robust PCA as an example of an additive decomposition that could be analyzed using the tools here. There exist analyses of robust PCA with missing entries, but all that can be shown is that observed portion of the sparse component is recovered.  Unless I'm mistaken, it should be possible to prove a better bound than (12) on the quantity w_{\Omega,W}(S), using a fairly simple argument. For S of interest here, w_G(S) > 1, and so the bound (12) is larger than w_G(S) (often substantially larger). One can write (abbreviating \Omega as O and P_\Omega as PO):      w_{O,W}(S) = E_OE_W sup_{X \in S-S} < X, PO(W) >   (11)                         <= 2 E_O E_W sup_{X \in S} < X, PO(W) >                         =2 E_O w_G( PO(S) ),(*)Consider the natural metric space on S, with d(s,t) = ( E | < s - t, w > |^2 )^(1/2); projection onto \Omega is a contraction, with respect to this distance and so it does not increase the \gamma_2 function, hence,         w_G(PO(S)) <= K w_G(S), givingw_{O,W}(S) <= K' w_G(S) via (*) and the fact that \gamma_2 gives a tight bound on the expected supremum of a Gaussian process. Am I missing something? The authors should be careful about anonymizing their supplementary material for conference submissions.