Lifelong Learning with Non-i.i.d. Tasks
In this work we aim at extending the theoretical foundations of lifelong learning. Previous work analyzing this scenario is based on the assumption that learning tasks are sampled i.i.d. from a task environment or limited to strongly constrained data distributions. Instead, we study two scenarios when lifelong learning is possible, even though the observed tasks do not form an i.i.d. sample: first, when they are sampled from the same environment, but possibly with dependencies, and second, when the task environment is allowed to change over time in a consistent way. In the first case we prove a PAC-Bayesian theorem that can be seen as a direct generalization of the analogous previous result for the i.i.d. case. For the second scenario we propose to learn an inductive bias in form of a transfer procedure. We present a generalization bound and show on a toy example how it can be used to identify a beneficial transfer algorithm.
The paper presents some extensions to the Pentina and Lampert's PAC-Bayesian analysis of "Lifelong Learning" problems (ICML 2014) , where a learner must adapt to various tasks exploiting knowledge from previously seen ones. The main contributions are risk bounds dedicated to two scenarios where the observed task are not sampled independently from each other. Roughly speaking, the first scenario share similarities with domain adaptation (albeit the risk bound is given on an average of all possible domains, instead of on a specific target domain) and the second is quasi-identical (up to my knowledge) to distribution drift. In the first setting (Section 3), the authors cleverly reuse Ralaivola et al.'s chromatic PAC-Bayesian theory to represent dependencies between tasks. However, this result alone let me unsatisfied. I wonder to which extent this result can be useful to the ambitious "lifelong learning" problem the authors are interested in. At the end of Section 3, the authors open the way to use the result as a "quality measure" benchmark or to the conception of a learning algorithm. If provided, such empirical study could be a good way to enhance this work.The second setting (Section 4) is very similar to the "distribution drift" scenario. I would like the authors to compare their results with other ones in this area (e.g., Mohri and Medina's "New Analysis and Algorithm for Learning with Drifting Distributions", ALT 2012). I also think that this setting must be studied more deeply. Contrary to the author claim, I think that the assumption of Equation (18) is very restrictive: it is unlikely that *all* algorithms of a considered family would have *exactly* the same error on every two consecutive tasks. This assumption should be relaxed to become realistic (as in Mohri and Medina 2012). Moreover, the algorithm should be able to consider more than the two previous tasks.Finally, throughout the paper, I wondered why the authors choose this specific formulation of PAC-Bayesian theorems. While Theorem 1 tradeoff is very easy to understand, the obtained bound is less tight than the bound of McAllester ("PAC-Bayesian stochastic model selection", Mach Learn 2003), where the complexity term appears under a square root, while being "explicit" (see also the slightly tighter version of Germain et al.'s "Risk Bounds for the Majority Vote...", JMLR 2015). Consequently, I presume that the further bounds for non-i.i.d. tasks are not as tight as they can be. I think this must be discussed in the paper.Some typos and minor comments: - There is a small error in Theorem 1 proof: When applying Hoeffding's lemma (Equation (3) of the Supplementary Material), there should be a "4" before lambda^2, because the subtraction between losses lies in [-1,1]. I also think that the same error appears at Equations (11), (25) and (26) of the main paper. - Line 89: The dot is misplaced - Supplementary Material, Line 78: Fro -> For - For the reader's benefit, it is preferable to cite the more recent version of Ralaivola et al.'s "Chromatic PAC-Bayes bounds for non-iid data" (JMLR 2010 instead of AISTATS 2009) - I personally prefer when the definitions, theorems, lemmas, etc share the same counter.