Recovering sparse conditional independence graphs from data is a fundamental problem in machine learning with wide applications. A popular formulation of the problem is an 1 regularized maximum likelihood estimation. Many convex optimization algorithms have been designed to solve this formulation to recover the graph structure. Recently, there is a surge of interest to learn algorithms directly based on data, and in this case, learn to map empirical covariance to the sparse precision matrix. However, it is a challenging task in this case, since the symmetric positive definiteness (SPD) and sparsity of the matrix are not easy to enforce in learned algorithms, and a direct mapping from data to precision matrix may contain many parameters. We propose a deep learning architecture, GLAD, which uses an Alternating Minimization (AM) algorithm as our model inductive bias, and learns the model parameters via supervised learning. We show that GLAD learns a very compact and effective model for recovering sparse graphs from data.
This paper proposes an approach to data driven edge recovery for sparse gaussian mrfs. The authors propose an AM procedure for solving the l1 regularized maximum likelihood which can be unrolled and parameterized. This method is shown to converge faster at inference time than other methods and it is also far more effective in terms of training time compared to an existing data driven method. The authors provide a theoretical analysis which explains how the AM procedure should succeed and some insights on how it can potentially converge better using an adaptive model (motivating the learning part).
Experiments:
Overall the experiments demonstrate the method is superior. I have a few comments/concerns however:
-Scalability to larger graphs. The graphs used here a relatively small. I would like to see how well this scales to larger graphs at least in principal if not experimentally. Is there any issues that make this difficult? For example more iterations might be needed for convergence and this becomes problematic for learning. Can this already compete  with large scale methods like BigQUIC.
-Are the training graphs always the same distribution as the test graphs (e.g. in terms of the sparsity level)? It would be good to evaluate how well the model works when the training conditions differ to testing, since applying it to real data would require this gap.
-Closely related to the above if I have understood correctly all the experiments including the gene networks are on synthetic data, it would be good however to see if synthetic data can help generalize to real data.
- How many iterations are used to train the model? Is the number of iterations ever more at inference than training? It seems the NMSE is increasing after hitting a bottom I am wondering if that is related to mismatch in the number of iterations in test/train
- (minor) the authors compare wall clock time per iteration in Table 2 however their method converges much faster, it would be good to also see the overall clock time for each method after some reasonable stopping crieria, to show how big the overall gain is. 
Related Work:
The overview of sparse graph recovery for Gaussian random variables is good and concise. However I found the high level motivations given in Introduction/Sec 3 are similar (at times even the wording) to those of Belilovsky et al 2017 which introduced/motivate the data driven approach to this problem. Although this reference is used in the experimental section, it would be appropriate to clarify the difference/contribution compared to this work in the Intro, Sec3, and/or related work as a naive reading of the paper incorrectly suggests it is the first to consider a data driven approach to this problem.
Other comments:
- In equation (9) \Theta^{*(i)} should there be an (i) index there, I suspect this is a typo but  if it is not can the authors explain how the target differs for different (i)
- The data generating process described in Sec 5, how does it assure SPD, the described procedure (sampling off diagonal entries U(-1,1) then randomly setting zeros) does not appear to me to assure SPD without further constraints.
- I appreciated Appendix C10/C11 the overview of other attempts to parametrize the inference procedure
Overall I found this work relevant, the formulation well motivated, and potentially of high impact for the community working on inference of sparse conditional independence structure. I would give the score at the moment between weak accept and accept. There is a few points which I would like the authors to clarify or correct in their rebuttal and I would be happy to increase my score.
-----Post Rebuttal
The authors have addressed my primary concerns and revised the text, I am thus increasing my score. I recommend the authors also itemize the main changes in the text from the initial manuscript as they are not easy to find right now using the openreview revision comparison system (which seems to be broken). For example its not clear if the results shown within text are in the new manuscript.