Tight convex relaxations for sparse matrix factorization
Based on a new atomic norm, we propose a new convex formulation for sparse matrix factorization problems in which the number of non-zero elements of the factors is assumed fixed and known. The formulation counts sparse PCA with multiple factors, subspace clustering and low-rank sparse bilinear regression as potential applications. We compute slow rates and an upper bound on the statistical dimension [1] of the suggested norm for rank 1 matrices, showing that its statistical dimension is an order of magnitude smaller than the usual 1 -norm, trace norm and their combinations. Even though our convex formulation is in theory hard and does not lead to provably polynomial time algorithmic schemes, we propose an active set algorithm leveraging the structure of the convex problem to solve it and show promising numerical results.
This paper proposes an atomic norm for estimating low-rank matrix whose factors are sparse. The authors show that the proposed atomic norm can be expressed as additive decomposition of a given (or parameter-) matrix into a sum of low-rank matrices, each of which is supported on at most k rows and q columns. The authors show a "slow rate" for the denoising performance of the proposed norm and compare an upper bound for it with a lower bound for another norm defined as a convex combination of the trace norm and the L1 norm. The method is applied to a drug-target-interaction-prediction problem and shows a AUC score better than the one predicted in the previous paper.The strength of the paper is that it tackles a problem that have wide applications (bilinear sparse regression, subspace clustering, sparse PCA) with a promising approach. The theory and experiments also seem to show the advantage of the proposed atomic norm over simply using the trace norm, L1 norm, or the convex combination of them.The weaknesses of the paper are as follows. - the theories (denoising bound and statistical dimension) are both limited to rank one. - optimization algorithm could be improved (it uses a generalization of sparse PCA algorithm [22] to compute the dual norm). Could the observation that the proposed norm reduces to the k-support norm for m_2=1 be used to develop a fast algorithm as in [2]? - Details of the drug-target-interaction experiment are missing. How were the training and test sets seprated? How sensitive was the result to the choice of k and q. - There were many typos. Sometimes it was hard to understand what the authors mean. For example, "By contrast to sequential approaches that estimate the principal components one by one [12], this formulation requires to find simultaneously a set of complementary factors." (line 164). What do you mean by complementary?Additional comments: - line 306: K cannot be PSD because K is in general asymmetric. - Does the theoretical guarantee in [22] extend to the proposed asymmetric generalization of TPI algorithm?