Split LBI: An Iterative Regularization Path with Structural Sparsity
An iterative regularization path with structural sparsity is proposed in this paper based on variable splitting and the Linearized Bregman Iteration, hence called Split LBI. Despite its simplicity, Split LBI outperforms the popular generalized Lasso in both theory and experiments. A theory of path consistency is presented that equipped with a proper early stopping, Split LBI may achieve model selection consistency under a family of Irrepresentable Conditions which can be weaker than the necessary and sufficient condition for generalized Lasso. Furthermore, some 2 error bounds are also given at the minimax optimal rates. The utility and benefit of the algorithm are illustrated by applications on both traditional image denoising and a novel example on partial order ranking.
This paper is interested in recovering a (non-sparse?) signal. from noisy obervations. The used loss-function ressembles the lasso but the authors clearly state the differences. An optimisation algorithm is proposed. The work provides many usual theoretical guarantees.
Quite like the paper, funny application (not a really important scientifically motivated question though) towards the end. More exploiting the image denoising to compare to state-of-the-art might be more useful in our community. Lots of theoretical results. Clear which should be noted, it is not always the case at all in related works! A companion empirical study would be welcome.  Small questions:  - l13: explain in more details the meaning/interpretation (e.g. related to practical application) of $ m $ and $ p $. Can see noise in $ \beta^{\star} $. A noisy version of $ \gamma^{\star} $? Still, it is neither observed nor measured.  - l25: meaning of ADMM?  - l37: 'structural' sparsity, please explain.  - l41-44: rephrase please  - l50-51: relationship to Champion et al. JSPI 2014?  - l54 bias-free Oracle: lasso or LBI?  - l66: $ (D_{1}\beta)_{p} = 0 $ ??  - l72: AUC of ROC? What about Precision vs Recall? Depends on sparsity in recovered structure...  - Fig. 1 and corresponding text: include that the larger the $ \nu $, the smaller the penalisation to help with the interpretation. (e.g. l165/166)  - l107: is the restriction to $ L $ a big one? In which setting? Can be assessed on data?  - l109: suprised you don't need the dependence of the Hessian on $ \beta $ and $ \gamma $ since gradient with respect to these parameters are its components. Can you clarify please?  - eqn (2.5) definition of $ A $ is a repeat of line 128  - l134 typo: irrEpresentable; l135: which proveS...  - eqn (2.8) and ones after l152: what do these limits represent? Interpretation??  - end of 2.2 rephrase correctly to mention next section of what follows  - Theorem 2 condition on $ \kappa $ (l189 and 190) are these conditions always compatible??  - l195: comment on rate  - l196: what happens to the last term on the rhs of the equation? (idem in Theorem 3)  - l212: how do you know $ r' $ is small? Hopefully not because of $ n $?!  - l220: 'reset' ?  - l226: evolves: increases??  - l227: can't see picked up random noise on Fig. 2...  - Fig. 2 left: magnify!!