Lifelong Learning with Weighted Majority Votes
Better understanding of the potential benefits of information transfer and representation learning is an important step towards the goal of building intelligent systems that are able to persist in the world and learn over time. In this work, we consider a setting where the learner encounters a stream of tasks but is able to retain only limited information from each encountered task, such as a learned predictor. In contrast to most previous works analyzing this scenario, we do not make any distributional assumptions on the task generating process. Instead, we formulate a complexity measure that captures the diversity of the observed tasks. We provide a lifelong learning algorithm with error guarantees for every observed task (rather than on average). We show sample complexity reductions in comparison to solving every task in isolation in terms of our task complexity measure. Further, our algorithmic framework can naturally be viewed as learning a representation from encountered tasks with a neural network.
The paper presents a learning algorithm that can learn many (different but related) prediction tasks by transferring knowledge obtained from previously solved tasks. The algorithm receives the tasks sequentially and choose to either: - build a majority vote predictor by weighting previously learned predictors; or - learn a new predictor "from scratch". The algorithm is inspired by VC-dimension regret bounds and is designed to obtain theoretical generalization and sample complexity guarantees. However, no empirical experiments are performed.
From my very personal point of view, the lifelong learning paradigm is a vague concept and is sometimes evoked for studying different scenarios that can be named otherwise (like transfer learning). However, I think that the framework studied here (from Balcan et al., 2015) is a very pertinent "lifelong problem". The authors present an honest work in the right direction. The proofs are not trivial but, as I explain below, the contribution appears to me insufficient for NIPS.  The risk bound minimized by the learning algorithm may be very high, as it relies on the VC-dimension of the predictors. Also, the role of the constant C in Theorem 1 is never discussed, and little is said about the way to handle it in the algorithm. Therefore, I am unconvinced that the algorithm will lead to a successful empirical procedure. I consider that the authors should provide empirical evidences that the approach is sound (at least on a synthetic dataset).   It is also likely that a tighter bounding technique (such as Rademacher or PAC-Bayes, both used by some of the cited works) provides better theoretical guarantees and/or empirical results. If there is a particular reason to rely on VC-dimension, the authors should discuss their choice.  In the introduction (Lines 47 to 50, and 80-81), the authors claim that, as opposed to Balcan et al. (2015), their algorithm learns "with weighted majority votes rather than linear combinations over linear predictor" and that "the shift from linear combinations to majority votes introduces a stability to the learned ensemble that allows exploiting it for later tasks." However, this comparison is not explicitly discussed later on. The meaning of the claim is unclear to me; In general, a linear combination of predictors can be expressed as a majority vote with a simple reparameterization.  ***  Post-rebuttal comments:  Considering the authors’ rebuttal, I raised my "technical quality" score a bit.   I still think that the paper would benefit from some experiments, in order to assess if Algorithm 1 would make an empirically good learning algorithm. It would also help to answer practical questions like: - Is the "\gamma-effective dimension" assumption realistic?  - Does the number of required labels for each task will be reasonable? Or should we manually "override' the constant C to make the algorithm work empirically?   The authors argued that we must consider the paper as a theoretical work, and leave these questions for future work. This is defendable. Personally, I feel that a work presenting a learning algorithm without any empirical results (not even on a toy dataset) is a bit incomplete (sure, Balcan et al. did it beforehand, but in COLT, which is strongly theory oriented).   Finally, regarding the linear combinations vs majority votes misunderstanding, I think the distinction must be clearly specified in the paper. From my point of view, a linear *separator* is given by \sign( < v_i, x > ), albeit a linear *regressor* is given by < v_i, x >. Moreover, a majority vote ca be a aggregation of either classifiers or regressors.