Non-convex Statistical Optimization for Sparse Tensor Graphical Model
We consider the estimation of sparse graphical models that characterize the dependency structure of high-dimensional tensor-valued data. To facilitate the estimation of the precision matrix corresponding to each way of the tensor, we assume the data follow a tensor normal distribution whose covariance has a Kronecker product structure. The penalized maximum likelihood estimation of this model involves minimizing a non-convex objective function. In spite of the non-convexity of this estimation problem, we prove that an alternating minimization algorithm, which iteratively estimates each sparse precision matrix while fixing the others, attains an estimator with the optimal statistical rate of convergence as well as consistent graph recovery. Notably, such an estimator achieves estimation consistency with only one tensor sample, which is unobserved in previous work. Our theoretical results are backed by thorough numerical studies.
after rebuttal: As the irrelevance of the initialization is one of the main contributions of this paper, it is strange to me that the authors adopted the same initialization as previous papers. This really raises a big concern on the work.============ In my opinion, this is a technically sound paper with nontrivial results. Following a recent thread of research on alternating minimization applied to different statistical problems, the authors established interesting results on recovering precision matrices from tensor normal distributions, and hence can be viewed as another manifestation of the power of alternating minimization to statistical problems. Theorem 3.1, though the simplest result in this paper, presents the core idea and hence is perhaps the most important observation in this paper for followup researches. The proofs of other theorems are more technical, but are clear enough for me to follow, and contain no unfixable problems. The proof of the last theorem is missing. The following are some errors or drawbacks I found in this paper:(1) The authors claim that one sample suffices to achieve strong statistical guarantees, but the current proof does not support such strong claim: In line 616 of Supplementary Material, the Frobenius norm of the error is only bounded for sufficiently large n.This problem may be avoided by considering fixed n and growing dimensions such that the the radius in the definition of $\mathbb{A}$ vanishes. For example, the scaling law considered in line 294 of the main text satisfies the above requirement, and therefore the conclusion is still valid. Only some parts of the proof need to be slightly modified.(2) In Supplementary Material there are some small errors in proofs: 2.1) Line 607, what is the "boundary" of $\mathbb{A}$? $\mathbb{A}$ is already a sphere-like set, so I see no point talking about the boundary of $\mathbb{A}$.2.2) Line 661, a typo in the second term of the first equality.2.3) Line 669, I don't see how the inequality here is applied. What I understood is to simply use sub-multiplicativity of the Frobenius norm and 667 follows.2.4) Line 699, "primal"-dual witness.2.5) In the proof of Lemma B.1, the constant  is not explicitly defined. I would suggest to include the definition of  in the statement of the lemma, so as to highlight the fact that the bound in Lemma B.1 depends on || - *||F .2.6) In the main text the authors state that their proof relies on Talagrand's concentration inequality. Where is the inequality used? Lemma C.2 is not Talagrand's concentration inequality; Gaussian concentration is a classical result that can be simply obtained through, for example, Gaussian log-Sobolev inequality.(3) Line 303 in the main text, the authors state that their bound is minimax optimal. Minimax with respect to what class?(4) Line 323, the Hessian := *-1 ? *-1?