Online Gradient Boosting
We extend the theory of boosting for regression problems to the online learning setting. Generalizing from the batch setting for boosting, the notion of a weak learning algorithm is modeled as an online learning algorithm with linear loss functions that competes with a base class of regression functions, while a strong learning algorithm is an online learning algorithm with smooth convex loss functions that competes with a larger class of regression functions. Our main result is an online gradient boosting algorithm that converts a weak online learning algorithm into a strong one where the larger class of functions is the linear span of the base class. We also give a simpler boosting algorithm that converts a weak online learning algorithm into a strong one where the larger class of functions is the convex hull of the base class, and prove its optimality.
The paper extends the theory of boosting to the online learning setting, proposing strong learning algorithms, building up from weak learners. First of all, I want to say that the topic of online boosting is quite fascinating. Theory of boosting has been of great significance to both statistics and machine learning community over the past two decades. Recently, there has been a couple of papers which has tried to extend boosting for binary classification to the online setting. In fact, the latest paper (by Satyen Kale et al.) won the best paper award at ICML 2015! Significance- As discussed above, the significance of online boosting is beyond question.Originality- As far as I know, no one has published work on extending boosting for regression to the online setting. Definitely original.Quality- Assuming the proofs are correct (I have not checked the analysis of Algo.2 and I have a major concern about the regret bound of Alg.1, which I will come to later), the paper is definitely of good quality.Clarity- In my opinion, the main drawback of the paper is the presentation. I will list as many concerns as I can recollect. 1. Discussion about weak and strong learners- The paper directly gives a definition of weak learning algorithm and strong learning algorithm. What are the intuition behind them? How do they relate to weak and strong learning for regression in the offline case? I am much more familiar with boosting for classification. I know that boosting for regression is approached more from a statistical viewpoint (as greedy forward model selection). Is there a weak and strong learning equivalent in the offline setting for regression?2. Line 117-125- I am slightly perturbed that just 1 paragraph sums up online boosting for regression. It is a very novel concept and in my opinion, requires much more discussion. Why is the strong learner defined for richer regression function class F' and richer loss function class C'? Specifically, why is the loss class changing? What is the equivalent in offline setting? 3. Line 136-138- I did not understand why we cannot have uniform regret bound for all functions in span(F). In fact, if the paper is the online equivalent of Zhang and Yu's paper (as stated by the authors), then the strong learner should converge to the best linear combination of base functions (i.e., min over span(F)).4. The richer loss function class C not only needs to be convex but also Lipschitz and more importantly, smooth. This was not mentioned in the abstract, where it seems the results will hold for all convex functions. 5. Alg.1- Why not have some intuition leading to Alg.1? I had to read the proof to even start understanding what is going on in the algorithm. It is not necessarily a criticism, but I always feel discussions after a very technical algorithm is helpful. As an example, see Kale's paper on online boosting. They lead to their algorithm in a very natural way. 6. Theorem 1- I have a big concern about the theorem. First of all, it seems for the regret to go to 0, N -> \infty. However, for very large N, does the algorithm remain efficient? This is because in step 8, d-dimensional vectors y^i are created for all copies of A (i.e. total of N). Will not soon become too large? More important question- The claim is that average regret goes to 0 as N and T-> \infty. It is not clear what is the rate at which the regret goes to zero. Considering the first term in the regret will go to zero only for \eta= O(log(N)/N) (though it is not clear to me at what rate), the second term is then going to 0 at rate O(log(N)/N), which is slower than even linear (in N) rate. A major advantage of boosting is the rate at which training error goes to zero. Even for online boosting (Kale et al.'s paper), they have error going to zero exponentially fast with N (effectively agreeing with offline boosting, where training error goes to zero exponentially fast with N). So combined with the fact that the regret is not even uniform over the linear span, I would like some clarification of the issue.7. In Alg.1, why cannot we just set B=\etaND? Why need to take a min over some other value? I did not see what it was adding to the algorithm. Line 284 shows the equality and the upper bound in line 297 will come from the Taylor expansion. Though taking a smaller B will give smaller regret, on the other hand, B= \etaND will get rid of the extra \eta^2 \beta B^2 term and make it more readable, isn't it? 8. Why will we care about competing with the convex hull of H? What is the offline equivalent? I can understand competing with best in span(F) but why convex hull?8. In Theorem 2., what does dependence of this regret bound on N is optimal mean? Should I interpret the sentence as : No strong learning algorithm can have a tighter regret in terms of N, against all adversary? There are more places where I felt more discussion could have been added. While I want to emphasize that I strongly believe this is a good piece of work, I think it will be greatly beneficial to the machine learning community if the paper goes through a solid round of editing before acceptance.