On Sparse Gaussian Chain Graph Models
In this paper, we address the problem of learning the structure of Gaussian chain graph models in a high-dimensional space. Chain graph models are generalizations of undirected and directed graphical models that contain a mixed set of directed and undirected edges. While the problem of sparse structure learning has been studied extensively for Gaussian graphical models and more recently for conditional Gaussian graphical models (CGGMs), there has been little previous work on the structure recovery of Gaussian chain graph models. We consider linear regression models and a re-parameterization of the linear regression models using CGGMs as building blocks of chain graph models. We argue that when the goal is to recover model structures, there are many advantages of using CGGMs as chain component models over linear regression models, including convexity of the optimization problem, computational efficiency, recovery of structured sparsity, and ability to leverage the model structure for semi-supervised learning. We demonstrate our approach on simulated and genomic datasets.
This paper explores how sparse conditional gaussian graphical models (sparse CGGMs) can be used as components of chain graphs. The authors discuss the qualitative and quantitative advantages of using CGGM formulation compared to using sparse multivariate regression with covariance estimation (MRCE) for two and three layer Gaussian chain graph. They go further to show how the framework can easily be extended to handle missing data in the middle layer as semi supervised learning. On their simulation data they show clear advantage of making use of the using CGGM for both network structure reconstruction and for prediction accuracy. Pros:The paper is clearly written. It shows how sparse CGGMs can be embedded in a multi layer chain graph structure and covers both structure recovery and inference in the resulting models. The models have intuitive interpretation in the context of genomic data. The improvement in both structure learning and prediction compared to using linear regression components are clear in the synthetic datasets considered.Cons: In general, the authors build heavily on previous works that developed sparse CGGMs (Shon & Kim 2012, Wytock & Kolter 2013). The review style of writing makes it hard to pin down directly from the text what was previously done and what is novel. The authors should state clearly the contributions.The results/evaluation for real life genomic data evaluation appear limited, cryptic and unconvincing. This makes the method overall less convincing especially as a way to tackle the bio-medical analysis problem they pose. (a) How are the dimensions/features selected? The numbers (e.g. 1000 SNPS, 200 expression "traits"?) seem unrealistic/irrelevant for the general task they offer to tackle originally (millions of SNPS, thousands of genes etc.)  (b) How is performance evaluated and what is the meaning of the prediction error values in Table 1? How is "structure" being scored in this setting?? (c) There are no error bars for these results (Table 1) (d) Contrary to the claim in the main text (Line 422) the improvement over LR models appears to be minute to non existing. The authors comment about the ability to deduce dependency between input features (p. 5 top) but this ability is not explored in their experimental evaluations.Other comments:Line 063: "more appropriate" than what?Line 74-75: "We show that our approach  can reveal the module structure ...in different types of genomic data effectively" - this is actually shown only for synthetic data experiments. The relation to real genomic data is not clear.Line 158: "recently it has been argued" Where? please give a citation.Line 204: Many of the multi task learning works do not assume the different "tasks" or outputs share the same values from the input features, which is the underlying assumption in the CGGM model. The authors should clarify that point to put their work in the right context.Figure 2: It seems x6 is irrelevant for the points been made in both Fig2a and 2b. If so consider removing it.Line 261: "It is well-known that...." is rather a strong statement that implies the relations described always hold. Consider revising (e.g."in many cases...") and adding an appropriate reference(s) that clearly establish the soundness of the claim.