Coordinate Descent with Bandit Sampling
Coordinate descent methods usually minimize a cost function by updating a random decision variable (corresponding to one coordinate) at a time. Ideally, we would update the decision variable that yields the largest decrease in the cost function. However, finding this coordinate would require checking all of them, which would effectively negate the improvement in computational tractability that coordinate descent is intended to afford. To address this, we propose a new adaptive method for selecting a coordinate. First, we find a lower bound on the amount the cost function decreases when a coordinate is updated. We then use a multi-armed bandit algorithm to learn which coordinates result in the largest lower bound by interleaving this learning with conventional coordinate descent updates except that the coordinate is selected proportionately to the expected decrease. We show that our approach improves the convergence of coordinate descent methods both theoretically and experimentally.
The paper introduces a new adaptive coordinate descent algorithm, which selects the coordinates based on a measurement of marginal decrease. The main idea is to measure the marginal decrease of each coordinate with the help of dual residues. The greedy algorithm consists of choosing the coordinate which provides the largest  marginal decrease. However, the greedy version requires the information of all the coordinates which is computationally expensive. To overcome this issue, an adaptive variant which updates the information in an online style has been introduced.   My major concern is about the theoretical analysis of the adaptive algorithm. In Proposition 2, the convergence rate of the algorithm is given under the assumption that the initial function gap f(x_0)-f* is smaller than some quantity Q. However, this assumption is unnatural and inappropriate due to the dependance between Q and f. It is not true that this condition will hold by scaling down the function f, because Q will also scale down. Moreover, Q depends on another highly non trivial quantity c, which depends on the functional property and probably the initial point. Thus, a more careful clarification is needed.   Besides this negative point in the theoretical analysis, I find the experimental section unconvincing. In particular, the parameter choices seems very arbitrary, for example the regularization parameter. Moreover, it is not clear how the algorithm's key parameters are selected like \epsilon and E. For ridge regression, E is set to be n/2 and for other cases E is d/2. There might be a huge gap between the feature dimension d and the number of samples n. It sounds like taking the best value that works without any theoretical reason.  Overall, the idea of using the marginal decrease as a guidance of coordinate selection is very interesting, but I do not support publication of the paper in the current state due to the weakness in both theoretical and the experimental results.   Some additional remarks: 1) The l1 regularization is not L-bounded unless taking L=infinity  2) Some of the plots do not include any algorithm proposed in the paper, for example in logistic regression on dataset w8a or a9a in Figure 3.   EDIT: I thank authors for addressing may concerns. Even though I still find the theoretical result a bit weak, I believe the paper provides interesting insight and ideas with outstanding empirical performance, as such, I raise my score accordingly.