On model selection consistency of M-estimators with geometrically decomposable penalties
Penalized M-estimators are used in diverse areas of science and engineering to fit high-dimensional models with some low-dimensional structure. Often, the penalties are geometrically decomposable, i.e. can be expressed as a sum of support functions over convex sets. We generalize the notion of irrepresentable to geometrically decomposable penalties and develop a general framework for establishing consistency and model selection consistency of M-estimators with such penalties. We then use this framework to derive results for some special cases of interest in bioinformatics and statistical learning.
This paper proves sparsistency for a class of 
      M-estimators with what is called ``geometrically decomposable penalty''. 
      Geometric decomposability is I believe a novel concept introduced 
      by the authors. It is similar to decomposability in the Negabhan et al. 
      sense but specially suited for sparsistency analysis, i.e. primal dual 
      witness. It seems that decomposable norms are also geometrically 
      decomposable, though the authors did not elaborate on the connections 
      between these two concepts. The paper is well written. The 
      concepts are clearly explained except for a few minor points. I am not 
      clear what is the $\Phi$-norm that suddenly appeared in line 223 and 
      Theorem 3.4. Also, it would be nice to remind the reader that $m$ is the 
      restricted strong convexity constant in the theorems. The results 
      look reasonable. I have not checked the proof in detail. I think, except 
      for the introduction of the geometric decomposability concept, the proof 
      follows the Wainwright 2009 Sharp Thresholds paper and is formulaic; the 
      authors should correct me if I am wrong here. The authors also use the 
      sparsistency results to prove consistency, the process of which I think 
      also follows standard proof techniques. The authors could 
      elaborate more on the results in section 4. For instance, is there any 
      aspect of the theorems that is especially significant or interesting? 
      Also, when one uses equation (3.3) to derive the irrepresentability 
      condition of the generalized lasso, how does that condition compare with 
      the condition of vanilla lasso? Does the condition imply any constraints 
      on the matrix D? Is there a way to intuitively interpret the 
      irrepresentability condition for the exponential family results? The 
      condition there seems to involve only the log-partition function 
      $A(\theta)$. The paper also studies sparse M-estimation with 
      homogenous linear equality constraints. The picture there seems a bit 
      murky however; linear equality constraints should improve the rate of 
      convergence since it provides additional information but the theorems 
      don't explicitly show such improvements. The experimental section 
      is OK. The lines in Figure 1.2 do not seem to line up that well to me. 
      Also, it'd be nice to have some experimental results on generalized lasso. 
      This is a fairly good paper. The sparsistency analysis does not 
      seem very difficult or novel; the results, though not too surprising, are 
      solid; and the notion of geometric decomposability introduced in the paper 
      should be useful. Some miscellaneous notes: * type on equation 
      (2.2), one of the ``A'' should be ``I'' * I am not sure why the 
      authors mentioned isotonic regression. Isotonic regression has linear 
      inequality constraints yes but it is a nonparametric method. There are 
      $O(n)$ parameters in isotonic regression so restricted strong convexity 
      will not hold I think; do the authors assert that their framework applies 
      to sparse isotonic regression as well?