Exclusive Feature Learning on Arbitrary Structures via 1,2 -norm
Group LASSO is widely used to enforce the structural sparsity, which achieves the sparsity at the inter-group level. In this paper, we propose a new formulation called "exclusive group LASSO", which brings out sparsity at intra-group level in the context of feature selection. The proposed exclusive group LASSO is applicable on any feature structures, regardless of their overlapping or non-overlapping structures. We provide analysis on the properties of exclusive group LASSO, and propose an effective iteratively re-weighted algorithm to solve the corresponding optimization problem with rigorous convergence analysis. We show applications of exclusive group LASSO for uncorrelated feature selection. Extensive experiments on both synthetic and real-world datasets validate the proposed method.
The paper proposes a new penalty term that can be used in learning a regression model;it's based on the idea that features are grouped into (possibly overlapping) groupsand the penalty term discourages the use of nonzero weights for several featuresfrom the same group.  The paper describes how to solve the resulting optimization problemand presents an evaluation on several synthetic and realistic datasets, demonstratingthat the proposed approach works better than several alternatives.The paper is clearly written and well organized, and seems to present an interestingincremental improvement on various earlier LASSO methods (which are mentionedin the related work section).Comments for the authors:"exclusive group LASSO" [caption of Fig. 1] -- should be "Exclusive" "Eq.(2)" [caption of Fig. 1] -- space missing; there are several similar occurrences elsewhere"G_g \in P({1, 2, ..., p}) denotes a set of group g" [page 2] -- I found thissentence unclear.  What exactly is the relationship between g and G_g?  Earlieryou say that a group g is a subset of {1, 2, ..., p}, and later you give anexample where G_g = {1, 2, 4}, so it would appear that G_g is also a subset of{1, 2, ..., p}; what then is the difference between g and G_g at all?"feature index u \in supp(g) will be non-zero" [page 3] -- I found this sentencea bit unclear as well.  What exactly is this feature index u, how is it chosenand what is its purpose?On page 3, there are several instances where groups are described using squarebrackets: G_u = [1, 2], G_v = [2, 3] and so on.  Why do you use brackets herewhereas on page 2 you used curly braces (e.g. "G_g = {1, 2, 4}")?  This makes itseem as if the groups were now real-number intervals, but probably that's notintended here."is a diagonal matrices" [page 4] -- should be "matrix"I was slightly surprised when I saw that your experiments on realistic dataall focus on using your improved LASSO (and other methods) for feature selection(and then applying SVM to train a classifier); when I read about the definitionof LASSO in [21], I was under the impression that its original purpose was toimprove the training of regression models, and your eq. (2) seemed to be alongthe same lines as well, so I thought that you would use some regression tasksto evaluate the method in your experiments.  It might be useful to add some discussionon why you use the method only for feature selection (in the context of a classificationtask), and not for any regression tasks.  Additionally, it might be good to addsome of the more traditional feature-selection methods for comparison (e.g. informationgain etc.).Above the first table on p. 7, you say "we list the mean and variance of classificationaccuracy", but it wasn't quite clear to me what the mean and variance are taken over.  Did you perform some sort of cross-validation or something like that?