Bilevel Learning of the Group Lasso Structure
Regression with group-sparsity penalty plays a central role in high-dimensional prediction problems. However, most existing methods require the group structure to be known a priori. In practice, this may be a too strong assumption, potentially hampering the effectiveness of the regularization method. To circumvent this issue, we present a method to estimate the group structure by means of a continuous bilevel optimization problem where the data is split into training and validation sets. Our approach relies on an approximation scheme where the lower level problem is replaced by a smooth dual forward-backward algorithm with Bregman distances. We provide guarantees regarding the convergence of the approximate procedure to the exact problem and demonstrate the well behaviour of the proposed method on synthetic experiments. Finally, a preliminary application to genes expression data is tackled with the purpose of unveiling functional groups.
The paper addresses the group lasso problem where the groups have to be learned (rather than being pre-supplied). This problem is cast as a (continuous) bilevel optimisation problem and the optimisation problem addressed by an algorithm devised by the authors. Experiments on synthetic data are given and support the usefulness of the presented method.  Groups are discrete features so the authors have to relax the problem by removing the integrality constraint on group membership indicator variables. We are not given an idea of how much is "lost" by relaxing, but given the difficulty of the original problem I have no objection to solving a relaxation of the original problem.  The paper is well-written (apart from many small errors - see below), the problem worth solving, the algorithm reasonable and the experimental results are positive. Using validation error to learn the groups is nice and straightforward. The empirical results given are also well chosen, particuarly the comparison with oracle GL. So I have difficulty in finding substantive criticisms of the paper. It's a typical NIPS submission: optimise to solve some machine learning problem.  SMALL PROBLEMS  21: "a priori structure of the data". I found this confusing: what is a priori can not be found in the data.  25: we are only told later what \theta is so it is confusing here.  100: arbitrary -> arbitrarily  111: iterates -> iterations  135: the solve -> to solve  212: as less informative -> as uninformative  246,247: change to "...  Q sufficiently permits reaching"  247: stress out -> stress  250: "Since a group ..." This sentence needs rewriting.  Refs: Fix capitalisation: e.g. Lipschitz, Bregman  AFTER DISCUSSION / FEEDBACK  I remain positive about the paper!