Elementary Estimators for Graphical Models
We propose a class of closed-form estimators for sparsity-structured graphical models, expressed as exponential family distributions, under high-dimensional settings. Our approach builds on observing the precise manner in which the classical graphical model MLE "breaks down" under high-dimensional settings. Our estimator uses a carefully constructed, well-defined and closed-form backward map, and then performs thresholding operations to ensure the desired sparsity structure. We provide a rigorous statistical analysis that shows that surprisingly our simple class of estimators recovers the same asymptotic convergence rates as those of thè 1 -regularized MLEs that are much more difficult to compute. We corroborate this statistical performance, as well as significant computational advantages via simulations of both discrete and Gaussian graphical models.
I personally view this paper quite positively. I feel that the paper was well written, assumptions and theorems clearly stated, and (interestingly) the proofs quite simple and well written. On reading the reviewer discussion and author feedback, there seemed to be two main issues arising(1) Too similar to earlier papers? I think not - even if the high-level idea is similar, the devil is in the details, which need to be fleshed out to get the right rates of convergence, good performance in the experiments, and so on. If I gave someone capable the earlier paper and asked them to derive the kinds of results seen in this one, it would not be "straightforward" to extend those results - it really would need some hard work.(2) Too harsh assumptions? This might be true but does not warrant rejection - the idea of closed-formed estimators for these problems is *important* and the community deserves to be aware of these advances, especially because they have been shown to work very well practically in this paper - it really is a game-changer compared to graph lasso or even neighborhood estimation which needs lasso-solvers. I do think that further research will lighten the assumptions used (or get a better understanding of them).The experiments are very promising, this stuff actually seems to work fast and well, and so it is possibly a case of a gap between what works and what's provable. I suspect future theory developed by other people will weaken these assumptions - people will want to work on this or extend it etc because it seems to work well.In summary, I clearly stand on the side of acceptance.