Learning Sparse Gaussian Graphical Models with Overlapping Blocks
We present a novel framework, called GRAB (GRaphical models with overlApping Blocks), to capture densely connected components in a network estimate. GRAB takes as input a data matrix of p variables and n samples and jointly learns both a network of the p variables and densely connected groups of variables (called 'blocks'). GRAB has four major novelties as compared to existing network estimation methods: 1) It does not require blocks to be given a priori. 2) Blocks can overlap. 3) It can jointly learn a network structure and overlapping blocks. 4) It solves a joint optimization problem with the block coordinate descent method that is convex in each step. We show that GRAB reveals the underlying network structure substantially better than four state-of-the-art competitors on synthetic data. When applied to cancer gene expression data, GRAB outperforms its competitors in revealing known functional gene sets and potentially novel cancer driver genes.
This paper presents a novel method GRAB that jointly learns the network structure and assignment of variables into potentially overlapping blocks. The authors also proposed a reliable optimization algorithm with the block coordinate descent method. Although presented in the context of Gaussian graphical models, GRAB can be generalized to learning of other types of networks. The proposed method is also illustrated in simulations and a real data application to demonstrate its usefulness in uncovering meaningful block structures.   Overall it is an interesting and important work to integrate network estimation and community detection into one unified problem. The optimization is done seamlessly. However, there are also some issues to be addressed, mainly regarding the identifiability of the block memberships. 
 (1) In this framework, the entries in Z are within [-1, 1]. In the context of stochastic block model, there is also an assignment matrix Z with entries {0, 1} (see Rohe et al 2011 Spectral clustering and the high-dimensional stochastic blockmodel). How does Z here (in the special binary case) compare with the Z matrix in Rohe et al. 2011?  (2) In section 2.2, could you explain how the constraint |Z|_F \le beta prevents all regularization parameters from becoming zero?  (3) In section 2.2, the constraint (c) |Z|_2 < tau is to prevent the case where all variables are assigned to one block. Why is it not allowed for all variables to be assigned to one block? It's possible that one wants to analyze the interaction network among genes from the same pathway (the same block). In the case of a chain graph, how many blocks are there?    (4) In section 4.2, the authors explain how to obtain blocks from the estimated Z matrix. The assignment of the block membership seems rather exploratory. In view of the problem formulation in (3) with the penalty matrix is lambda * (1 - (ZZ^T)_{ij}), there might be identifiability issue. That is, lambda_1 * (11^T - Z_1 Z_1^T) = lambda_2 * (11^T - Z_2 Z_2^T) for (lambda_1, Z_1) \ne (lambda_2, Z_2). Consider the example of a chain graph. In this case, how should one interpret the matrix Z and the following assignment of block memberships?     (5) In Section 3.3, equation (8): diag((W)) --> diag(W). Similar corrections should be made one line above equation (9). When introducing equation (11), lambda_i's are used as eigenvalues, whereas lambda was used previously as the penalty parameter. Maybe a different notation is better. In equation (13), the rightmost parenthesis is missing. Further, do you still need the constraint diag(W) \le 1 in equation (13)?   (6) In Section 4.2, some details on how the results are obtained are missing. For example, why and how are the between-block edges removed from further analysis? If this is the case, should similar procedure be done for the competing method? In particular, the comparison in Figure 5 seems to be unfair for the Glasso. It is true that GRAB directly returns the block assignment, however one may also recover the blocks based on the estimated adjacency matrix from Glasso. Have you tried building blocks for the Glasso? In that case, it might be fair to compare the estimated blocks for both methods.