summaries: In contrast to previous works on lifelong machine learning (LML) that put their focus on the supervised learning settings, this paper concentrates on the scenario that only a limited amount of data is available. The proposed method MAKO is mounted on the top of supervised LML model, without introducing additional knowledge based overhead, for better leveraging the unlabeled data. Labeling new data can be realized by using the data programming method which is supervised by the labeled data. The target of this paper is to design a SSL LML framework that minimizes the performance between using partially labeled data, and the upper-bound performance using fully labeled data. Several experiments on standard image classification data sets including MNIST, CIFAR-10 and CIFAR-100 are used to evaluate the the effectiveness of MAKO.  This paper propose a wrapper tool that mounts on top of supervised Lifelong Machine Learning (LML) frameworks, leveraging a well-known method data programming. The contributions of this paper can be summarized in three aspects. 1)  Adapting automatic label generation by semi-supervised learning/data programming to LML in some special scenario. 2) Implementing a LML wrapper that can accomplish some tasks under some restrictions. 3) Through detailed experimental results prove the superiority of its method. This paper proposes  data programming method, named Mako,  for semi-supervised continual learning. Mako automatically generates labels for unlabeled data with a set of weak labeling functions, each of the functions is trained on subset of training set with bootstrapping. Experimental on several datasets demonstrate the effectiveness of the proposed methods.  This paper presents an interesting idea of using data programming techniques to enable continual semi-supervised learning with limited labeled data. It proposes a stage-wise pipeline where probabilistic pseudo labels are first produced by a Snuba based Data Programming framework, then calibrates them by the temperature scaling, and finally inputs into the mounted Lifelong Machine Learning (LML) tools. Experiments show that the proposed framework achieves similar performance to fully supervised methods.
abstractText: Lifelong machine learning (LML) is a well-known paradigm mimicking the human learning process by utilizing experiences from previous tasks. Nevertheless, an issue that has been rarely addressed is the lack of labels at the individual task level. The state-of-the-art of LML largely addresses supervised learning, with a few semi-supervised continual learning exceptions which require training additional models, which in turn impose constraints on the LML methods themselves. Therefore, we propose Mako, a wrapper tool that mounts on top of supervised LML frameworks, leveraging data programming. Mako imposes no additional knowledge base overhead and enables continual semi-supervised learning with a limited amount of labeled data. This tool achieves similar performance, in terms of per-task accuracy and resistance to catastrophic forgetting, as compared to fully labeled data. We ran extensive experiments on LML task sequences created from standard image classification data sets including MNIST, CIFAR-10 and CIFAR100, and the results show that after utilizing Mako to leverage unlabeled data, LML tools are able to achieve 97% performance of supervised learning on fully labeled data in terms of accuracy and catastrophic forgetting prevention. Moreover, when compared to baseline semi-supervised LML tools such as CNNL, ORDisCo and DistillMatch, Mako significantly outperforms them, increasing accuracy by 0.25 on certain benchmarks.
references.authors: M. Baucum D. Belotto S. Jeannet E. Savage P. Mupparaju C.W. Morato
references.title: Semi-supervised deep continuous learning
references.venue: In Proceedings of the International Conference on Deep Learning Technologies,
references.authors: A. Bulat J. Kossaifi G. Tzimiropoulos M. Pantic
references.title: Incremental multi-domain learning with network latent tensor factorization
references.venue: In Proceedings of the AAAI Conference on Artificial Intelligence
references.authors: R. Caruana
references.title: Multitask Learning
references.venue: PhD thesis,
references.authors: Z. Chen B. Liu
references.title: Lifelong Machine Learning. Synthesis Lectures on Artificial Intelligence and Machine Learning
references.authors: M. Cuturi
references.title: Sinkhorn distances: Lightspeed computation of optimal transport
references.venue: Advances in Neural Information Processing Systems (NIPS),
references.authors: C. Finn T. Yu J. Fu P. Abbeel S. Levine
references.title: Generalizing skills with semi-supervised reinforcement learning
references.venue: Proceedings of the International Conference on Learning Representations (ICLR),
references.authors: Y. Gao Q. She J. Ma M. Zhao W. Liu A.L. Yuille
references.title: NDDR-CNN: Layer-wise feature fusing in multi-task cnn by neural discriminative dimensionality reduction
references.venue: In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,
references.authors: C. Guo G. Pleiss Y. Sun K.Q. Weinberger
references.title: On calibration of modern neural networks
references.venue: Proceedings of the International Conference on Machine Learning (ICML),
references.authors: K. He X. Zhang S. Ren J. Sun
references.title: Deep residual learning for image recognition
references.venue: IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
references.authors: A. Krizhevsky
references.title: Learning multiple layers of features from tiny images
references.venue: Technical report, University of Toronto,
references.authors: A. Kumar H. Daume
references.title: Learning task grouping and overlap in multi-task learning
references.venue: In Proceedings of the 29th International Conference on Machine Learning,
references.authors: S. Laine T. Aila
references.title: Temporal ensembling for semi-supervised learning
references.venue: Proccedings of the International Conference on Learning Representations (ICLR),
references.authors: Y. LeCun C. Cortes
references.title: Mnist handwritten digit database
references.venue: URL http://yann.lecun. com/exdb/mnist/
references.authors: Y. Lecun L. Bottou Y. Bengio P. Haffner
references.title: Gradient-based learning applied to document recognition
references.venue: Proceedings of the IEEE,
references.authors: S. Lee J Stokes E. Eaton
references.title: Learning shared knowledge for deep lifelong learning using deconvolutional networks
references.venue: In Proceedings of the International Joint Conference on Artificial Intelligence,
references.authors: B. Liu
references.title: Lifelong machine learning: a paradigm for continuous learning
references.venue: Frontier of Computer Science,
references.authors: Y. Lu A. Kumar S. Zhai Y. Cheng T. Javidi R. Feris
references.title: Fully-adaptive feature sharing in multitask networks with applications in person attribute classification
references.venue: In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,
references.authors: I. Misra A. Shrivastava A. Gupta M. Hebert
references.title: Cross-stitch networks for multi-task learning
references.venue: In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,
references.authors: C. Olivier S. Bernhard Z. Alexander
references.title: Semi-supervised learning
references.venue: IEEE Transactions on Neural Networks,
references.authors: S.J. Pan Q. Yang
references.title: A survey on transfer learning
references.venue: IEEE Transaction on Knowledge and Data Engineering,
references.authors: J.C. Platt
references.title: Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods
references.venue: Advances in Large Margin Classifiers,
references.authors: A. Rasmus H. Valpola M. Honkala M. Berglund T. Raiko
references.title: Semi-supervised learning with ladder networks
references.venue: Advances in Neural Information Processing Systems (NIPS),
references.authors: A. Ratner S.H. Bach H. Ehrenberg J. Fries S. Wu C. Ré
references.title: Snrokel: rapid training data creation with weak supervision
references.venue: Proceedings of the VLDB Endowment,
references.authors: A. Ratner C.S. De Sa Wu D. Selsam C. Ré
references.title: Data programming: Creating large training sets, quickly
references.venue: Advances in Neural Information Processing Systems (NIPS),
references.authors: P. Ruvolo E. Eaton
references.title: ELLA: An efficient lifelong learning algorithm
references.venue: Proceedings of the International Conference on Machine Learning (PMLR),
references.authors: P. Ruvolo E. Eaton
references.title: Active task selection for lifelong machine learning
references.venue: Proceedings of the AAAI Conference on Artificial Intelligence,
references.authors: J. Schwarz W. Czarnecki J. Luketina A. Grabska-Barwinska Y.W. Teh R. Pascanu R. Hadsell
references.title: Progress & compress: A scalable framework for continual learning
references.venue: In Proceedings of the International Conference on Machine Learning,
references.authors: B. Settles
references.title: Active learning literature survey
references.venue: Technical report, Department of Computer Science, University of Wisconsin-Madison,
references.authors: J. Smith J. Balloch Y. Hsu Z. Kira
references.title: Memory-efficient semi-supervised continual learning: The world is its own replay buffer
references.venue: In Proceedings of the International Joint Conference on Neural Networks,
references.authors: G. Sun Y. Cong X. Xu
references.title: Active lifelong learning with ”watchdog
references.venue: In Thirty-Second AAAI Conference on Artificial Intelligence,
references.authors: A. Tarvainen H. Valpola
references.title: Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
references.venue: Advances in Neural Information Processing Systems (NIPS),
references.authors: S. Thrun T.M. Mitchell
references.title: Lifelong robot learning
references.venue: In The Biology and Technology of Intelligent Autonomous Agents,
references.authors: P. Varma C. Ré
references.title: Snuba: automating weak supervision to label training data
references.venue: Proceedings of the VLDB Endowment,
references.authors: L. Wang K. Yang C. Li L. Hong Z. Li J. Zhu
references.title: Ordisco: Effective and efficient usage of incremental unlabeled data for semi-supervised continual learning
references.venue: In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR),
references.authors: S. Wang Z. Chen B. Liu
references.title: Mining aspect-specific opinion using a holistic lifelong topic model
references.venue: Proceedings of the International Conference on World Wide Web (WWW),
references.authors: J. Yoon E. Yang J. Lee S.J. Hwang
references.title: Lifelong learning with dynamically expandable networks
references.venue: Proccedings of the International Conference on Learning Representations (ICLR),
sections.heading: 1 INTRODUCTION
sections.text: Since 1995, researchers have been actively seeking machine learning (ML) frameworks that can better mimic the human learning process by retaining memories from past experiences (Thrun & Mitchell, 1995; Liu, 2017). Hence, in contrast to traditional isolated ML where knowledge is never accumulated, the concept of lifelong machine learning (LML) is defined as training ML models for continual learning over task sequences, with a knowledge base to store information that could be helpful in the future. Under this definition, LML can be seen as continual transfer learning (Pan & Yang, 2015) with memory or sequential multi-task learning (Caruana, 1997).
  Nevertheless, at the level of individual tasks, the current state-of-the-art of LML still remains largely supervised. A challenge that has rarely been mentioned is that labels can still be expensive and rarely available for each task, such that supervised training can hardly provide acceptable performance. For LML, performance is not only measured at task level, but also for the entire lifetime, with attributes such as resistance to catastrophic forgetting and space-efficiency of knowledge base.
  We take a more generic approach to semi-supervised lifelong learning and develop a technique that can wrap around any existing continual learning algorithm. Specifically, we propose to use data programming (Ratner et al., 2016b) to label new data under the supervision of a set of weak labeling functions trained from a limited number of labeled data points. Data programming has proven successful for isolated learning, and so this is the first work integrating it with lifelong learning.
  Our approach, called Mako, sits atop an existing lifelong learner to turn it into a semi-supervised learner. For each task, Mako takes as input a small labeled dataset plus an unlabeled dataset with no restriction on size, generates labels for the unlabeled data and then updates the lifelong learner supervised by both sets. This approach builds upon the widely used Snuba algorithm (Varma & Ré,
  2016), and we show that its theoretical guarantees still hold in the lifelong setting. Mako’s procedure leverages automatic hyperparameter tuning, data programming and confidence calibration, in order to adapt automatic labeling to constantly varying LML tasks. We show that Mako produces adequately high performance, in terms of both task accuracy and prevention of catastrophic forgetting, that approaches that of training on fully labeled data without costing extra knowledge base space overhead.
  We extensively evaluated Mako on various partially labeled LML task sequences generated from commonly used datasets including MNIST, CIFAR-10 and CIFAR-100 (LeCun & Cortes, 2010; Krizhevsky, 2009), while mounting on a diversity of supervised LML frameworks such as Deconvolutional Factorized CNN (DF-CNN), Dynamic Expandable Network (DEN) and Tensor Factorization (TF) (Lee et al., 2019; Yoon et al., 2018; Bulat et al., 2020). We compared the results to supervised learning on fully labeled data, as well as the same partially labeled data with existing semi-supervised LML tools: CNNL, ORDisCo and DistillMatch (Baucum et al., 2017; Wang et al., 2021; Smith et al., 2021). Empirically, we show that the performance of LML methods improves as giving more training data even with Mako labels, achieving at least 97% performance relative to LML methods trained on ground-truth labels and being able to beat existing semi-supervised LML tools by approximately 0.25 higher accuracy.
  Contributions Summary:
  1. Adapting automatic label generation by semi-supervised learning/data programming to LML in the scenario where labeled training data is expensive to obtain compared to unlabeled for all tasks. 2. Implementing a LML wrapper, namely Mako, for the scenario of expensive labels, such that, when given partially labeled data of limited size and compared to the current supervised state-ofthe-art with fully labeled data, each LML task sequence (1) requires significantly fewer labeled data to achieve high accuracy on individual tasks, (2) maintains similar resistance to catastrophic forgetting and (3) does not cost extra knowledge base storage. 3. Extensive experiments on lifelong binary and multiclass image classification on 3 commonly used datasets, including comparison to supervised and semi-supervised LML baselines, accomplishing very close performance to supervised LML on fully labeled data and significantly outperforming baseline semi-supervised LML frameworks.
sections.heading: 2 RELATED WORK
sections.text: Lifelong machine learning (LML) is a concept of continual learning among a sequence of tasks. Specifically, given multiple machine learning tasks arriving continuously, the LML framework accumulates the knowledge of previous tasks and utilizes it for future ones (Chen & Liu, 2016; Ruvolo & Eaton, 2013b). The method of retaining and adopting the knowledge base varies depending on the characteristics of tasks and learning models. For instance, a latent matrix is used to keep prototypical weights for linear models (Kumar & Daume, 2012; Ruvolo & Eaton, 2013a), and for deep neural networks, either weights of layers (Yoon et al., 2018; Lu et al., 2017; Bulat et al., 2020; Lee et al., 2019) or extracted features of layers (Rusu et al., 2016; Misra et al., 2016; Gao et al., 2019; Schwarz et al., 2018) are considered as the knowledge base to transfer. Besides research on models to transfer knowledge across tasks, the LML community has also been working on task order sorting for higher overall performance (Sun et al., 2018; Ruvolo & Eaton, 2013b). Nevertheless, all these works assume that each individual task training is supervised with ground-truth labels for the entire training set. In many real-life ML tasks, obtaining labels has been and continues to be expensive (Settles, 2009). Only a few works have addressed this issue on LML in very specific domains, such as continual sentimental analysis with words partially labeled (Wang et al., 2016) and reinforcement learning in a sequence of state spaces without explicit rewards (Finn et al., 2017). Different from previous work, Mako enables high performance in continual learning on generic ML tasks given partially labeled data, with the cardinality of labels restricted at a small number.
  To tackle expensive labels in ML, researchers have purposed various methods such as active learning (Settles, 2009) or semi-supervised learning (Olivier et al., 2006), which is an ML framework that takes both labeled and unlabeled data as inputs. The unlabeled data can directly assist the training, or can serve as a target data set yet to be labeled. In recent years, with the rising enthusiasm and need in deep learning, various methods have been used to assist semi-supervised learning with DNN models (Rasmus et al., 2015; Laine & Aila, 2016; Tarvainen & Valpola, 2017). One branch of
  semi-supervised learning, namely data programming or weak supervision (Ratner et al., 2016b), focuses on auto-generation of labels on an unlabeled dataset with the help of a small labeled dataset. Targeting this problem, researchers leverage weak labeling functions that each can label with an accuracy slightly higher than a coin flip. Then, ensembling algorithms are able to bag these weak labelers and produce highly accurate final labels. In other words, the weak labels supervise the training of the final ensembled generative model. There exists different implementations of such generative model training such as Snorkel (Ratner et al., 2016a).
  Semi-supervised learning is an underexplored problem in the LML setting; all the works mentioned above have addressed problems in isolated ML only. Early work in semi-supervised incremental learning, in which all tasks are trained simultaneously as batches of unlabeled data are incrementally observed by a classifier, has identified challenges pertaining to selection of the label generating model architecture and confidence calibration of the generated labels (Baucum et al., 2017), processes which Mako undertakes autonomously through automatic search for weak labelers before ensembling, and confidence calibration after labels are produced. Pioneering work in semi-supervised LML requires training of additional models for out-of-distribution detection (Smith et al., 2021) or generative replay (Wang et al., 2021). Conversely, by focusing on label generation through data programming, our approach achieves similar behavior through a pre-processing workflow that does not impose additional constraints on the LML approach or require additional knowledge base storage.
  For data programming tools, it has been a long standing issue to obtain weak labeling functions as input. One approach is manually designing the weak labelers by domain experts (Ratner et al., 2016b;a), while other researchers seek automatic labeler generation. For instance, Snuba (Varma & Ré, 2016) outputs weak supervisors with an accuracy guarantee on the final ensembled data programming model. Thus far, Snuba has been evaluated on various isolated ML tasks, with weak labelers in form of k-nearest-neighbors, decision stumps and logistic regressors, and has been shown to ace in natural language processing and image classification. It also succeeds in more complex applications such as classifying bone tumors in medical domain. In our work, we build a weak labeling function generation module in Mako based on Snuba, aiming to support a larger variety of labeler models for LML, while maintaining the theoretical accuracy guarantee of generated labels.
sections.heading: 3 PROBLEM FORMULATION
sections.text: LML: Given a sequence of ML tasks: T1, T2, . . . , Tm, where m is an unbounded integer, each task Ti has an underlying data space Xi and label space Yi. Ti has ni,train labeled training data (Xi,train, Yi,train) ∼ Di, such that Xi,train ∈ X ni,train i and Yi,train ∈ Y ni,train i , following some task-specific i.i.d. distributionDi. The task also has ni,test testing data (Xi,test, Yi,test) drawn from the same distribution.
  Traditionally, an LML model encounters the tasks in order. At Ti, it trains on the full set of (Xi,train, Yi,train) with supervised learning and obtain model fi : ⋃i j=1 Xi 7→ ⋃i j=1 Yi. Immediately after this training, it predicts the j-th task with Ŷij = fi(Xj,test) for each j ≤ i. There are two objectives:
  1. High test accuracy. Formally, denote testing accuracy of task Tj right after the training of task Ti on (Xi,train, Yi,train) as
  aij =
  ∑nj,test k=1 1(ŷk = yk)
  nj,test (1)
  with ŷk and yk the k-th label in Ŷij and Yi,test, respectively. One LML objective in the current state-of-the-art is to maximize the peak per-task accuracy up to task i defined as
  ãi = 1
  i i∑ j=1 ajj (2)
  Peak per task accuracy is defined on all tasks i = 1, . . . ,m , and shows the LML performance on the current training task on the basis of the accumulated knowledge.
  2. Low catastrophic forgetting. A critical aspect of LML is to prevent catastrophic forgetting of previous tasks. The resistance to catastrophic forgetting can be quantified by the average testing accuracy of all tasks encountered so far. For each task Ti, denote a second objective, final accuracy
  up to task i as
  āi = 1
  i i∑ j=1 aij (3)
  The second objective is to maximize āi for all i = 1, . . . ,m.The gap between peak per-task accuracy and final accuracy expresses how much knowledge of previous tasks LML method can preserve through the streams of tasks. Therefore, we define a third metric to be maximized, catastrophic forgetting ratio up to task i, as
  ci = 1
  i i∑ j=1 aij ajj
  (4)
  This metric is less than 1 if the LML model loses its performance on the earlier tasks, and it is greater than 1 if there is positive knowledge transfer from the later tasks to the earlier ones.
  Semi-supervised LML with data programming: The key challenge we want to address is the cost of labeling data in all LML tasks. That is, the number of available labeled training data is small. Therefore, we would like to split (Xi,train, Yi,train) in the traditional LML problem into disjoint (Xi,L, Yi,L, Xi,U ), where (Xi,L, Yi,L) is a set of labeled training data with restriction on size and Xi,U is a set of unlabeled data with no such restriction. In other words, |Xi,L| is small, e.g. |Xi,L| ≤ 150 for a 10-way image classification problem. Apparently, the LML model is now trained with semi-supervised learning. For task Ti, we would like to leverage data programming on (Xi,L, Yi,L, Xi,U ) to first train a generative model πi : Xi 7→ Yi. This model automatically labels Xi,U with Y πi,U = πi(Xi,U ). Then, the LML framework will be trained on (Xi,L, Yi,L, Xi,U , Y πi,U ). Denote the LML model immediately after this training as fπi : ⋃i j=1 Xi 7→ ⋃i j=1 Yi, and Ŷ πij = fπi (Xj,test) for each task j ≤ i.
  In this scenario, we redefine testing accuracy in Equation 1 as
  aπij =
  ∑nj,test k=1 1(ŷ π k = yk)
  nj,test (5)
  where the only difference is that ŷπk is the k-th label in Ŷ π ij instead of Ŷij . From Equation 5, we can then redefine metrics in Equations 2, 3 and 4 as
  ãπi = 1
  i i∑ j=1 aπjj (6) ā π i = 1 i i∑ j=1 aπij (7) c π i = 1 i i∑ j=1 aπij aπjj
  (8)
  Our research problem is to design a semi-supervised LML framework leveraging data programming that minimizes the performance between using partially labeled data, (Xi,L, Yi,L, Xi,U , Y πi,U ), and the upper-bound performance using fully labeled data, (Xi,L, Yi,L, Xi,U , Yi,U ), in terms of these three metrics while imposing no additional knowledge base storage overhead in the LML framework.
sections.heading: 4 MAKO: LIFELONG MACHINE LEARNING FRAMEWORK
sections.text: We implemented Mako, a semi-supervised LML framework that can be mounted on top of any existing LML tool, with (Xi,L, Yi,L, Xi,U ) as input for each task Ti. The automatic label generation of Y πi,U is done by data programming, consisting of generating weak labeling functions and ensembling a generative model. Nevertheless, different from data programming on isolated ML, due to the variation of tasks throughout LML, we need to actively adapt both the weak and the strong models to maximize the performance at each task. This is accomplished by automatic search on architecture and training hyperparameter search before weak labeler generation and confidence calibration after the ensembled model πi is trained. The following subsections explains the Mako workflow step-by-step, as illustrated in Figure 1.
sections.heading: 4.1 AUTOMATIC HYPERPARAMETER SEARCH
sections.text: Before producing weak labeling functions, two questions must be answered: (1) How do we choose the model architecture for our weak labeling functions? (2) How do we train these models after architecture is decided? Formally, these questions lead to the decision of architectural and training hyperparameters of labelers, which Mako accomplishes automatically.
  The only prior knowledge required is a search space, manually designed, of the architecture and training hyperparameters. For instance, given that the task is image classification, the search space of labeler architecture will include CNNs with the same input and output dimensions, but different configurations of hidden layers. Similarly, the training hyperparameters will be searched on different learning rates, batch sizes and number of epochs.
  The automatic tuning of weak labelers therefore traverses the search space until a satisfactory configuration is found. The stopping criterion is defined as follows: after training ten labelers with a given configuration of architecture and training hyperparameters on different bootstrapped subsets of (Xi,L, Yi,L), 9 out of 10 achieves an accuracy on the entire (Xi,L, Yi,L) above some threshold a. This criterion shows the generalization capability of sampled labelers and alternative criteria can be used, such as 8 out of 10 or average accuracy ≥ a. We accept such a configuration because we train the labelers on bootstrapped subsets of data during data programming, which will be further explained in 4.2. If no configuration in the search space is satisfactory, we decrease a by 0.05 and repeat the search. Hence, we can start from a sufficiently high a such as 0.9. For efficiency, Mako selects the configuration with the smallest number of training epochs among all the acceptable ones.
  There exists a efficiency-automation trade-off in this hyperparameter search. The less prior knowledge we have for the task, the larger search space we need to input to Mako and thus a larger time overhead. In contrast, more manual exploration on the task will lead to a smaller search space and therefore efficiency.
sections.heading: 4.2 GUARANTEED DATA PROGRAMMING WITH BOOTSTRAPPING ON TRAINING SET
sections.text: After deciding the architecture and training hyperparameters of weak labeling functions, Mako kicks off weak labeler training and ensembling of generative model πi for each task Ti. For the training process, we are inspired by Snuba (Varma & Ré, 2016) due to the fact that it has a theoretical accuracy guarantee in the final generative model. Nonetheless, in order to increase variance among the labelers, Snuba trains them on different selected subsets of features. This procedure does little favor in LML, where we might encounter complex tasks in the sequence that requires the entire feature space. For instance, in image classification, excluding pixels in training may lead to a failure in capturing important feature-target relationships, especially when we want to take advantage of local connectivity of CNNs. Therefore, Mako uses bootstrapping on (Xi,L, Yi,L) and trains each labeler with a sampled subset. The bootstrapping size can be either manually given or searched as a training hyperparameter.
  Besides bootstrapping on data, the rest of weak labeler generation process remains unchanged from Snuba in order to maintain its theoretical guarantee. Snuba has such a guarantee because it checks an exit condition on whether the committed labelers have labeled sufficient number of data points in Xi,L with acceptable confidence. If so, these labelers guarantee that the final generative model will have a learned accuracy on Xi,L close to the labeling accuracy on Xi,U . Formally, this guarantee can be stated as Proposition 1.
  Proposition 1 (Theoretical Guarantee of Snuba): Given h committed weak labelers, denote the empirical accuracies of all committed weak labelers on Xi,L as a vector ai,L,w. The generative
  model training outputs learned accuracies of these labelers on Xi,L as ai,L,g before ensembling. Moreover, the unknown true accuracies of the labelers on Xi,U after generative model training are denoted as ai,U,g , with ai,L,w, ai,L,g , ai,U,g ∈ Rh. We have a measured error ||ai,L,w−ai,L,g||∞ ≤ . Then, if each labeler labels a minimum of
  d ≥ 1 2(γ − )2 log (2h2 δ ) (9)
  data points in Xi,L with above some given confidence threshold ν for all iterations, we can guarantee that ||ai,U,g − ai,L,g||∞ < γ for all iterations with probability 1− δ. The proof of Proposition 1 from the original Snuba paper is included in Appendix A. We would like to show that our bootstrapping technique maintains this guarantee, as in Proposition 2.
  Proposition 2 (Theoretical Guarantee of Mako Weak Labeler Generation): Despite the modifications from Snuba, the Mako weak labeler generation process satisfies the theoretical guarantees described in Proposition 1.
  Proof of Proposition 2: Although our weak labelers are generated with (1) different model architectures and (2) sampling by bootstrapping on data instead of features, it still satisfies the definition of weak heuristics required by Snuba. That is, a function f : Xi 7→ Yi ∪ {abstain}, where abstain is a special label assigned to data points with a confidence lower than the threshold ν.
  Consequently, by Proposition 2, if Mako checks the same theoretical exit condition at each iteration, the final generative model satisfies the same theoretical guarantee as in Snuba. The hyperparameter search described in 4.1 can be understood as looking for labelers with high ai,L,w. Based on this guarantee, we will have ||ai,U,g − ai,L,w||∞ ≤ + γ with probability 1 − δ, and hence we can probablistically achieve high labeling accuracy ai,U,g .
  The weak labeling functions are then used to label both Xi,L and Xi,U and the labels are input for the training of generative model πi. We label Xi,L as well for confidence calibration, which will be explained in the following subsection.
sections.heading: 4.3 CONFIDENCE CALIBRATION AND THRESHOLDING
sections.text: Considering the variability and complexity of LML tasks, we believe the generative model labels can be further improved to approach the ground truth. Therefore, Mako takes an extra step to adjust the trained generative model πi by confidence calibration on the produced logits on Xi,U .
  Confidence calibration (Guo et al., 2017) is the process to align the confidence of labeling a data point to the accuracy, which is an estimation of the probability that the data point is labeled correctly. The procedure of confidence calibration is adjusting the logits in order to minimize the expected calibration error (ECE), formalized as
  min pπk,i,U
  Epπk,i,U [|Pr[y π k,i,U = yk,i,U | pπk,i,U = p]− p|] (10)
  where yk,i,U is the ground truth label of the k-th data point in Xi,U , yπk,i,U is its predicted label by the data programming generative model and pπk,i,U is the associated confidence. However, since the ground truth labels ofXi,U is unknown, this approach is infeasible. This is the reason for training the generative model under the supervision of weak labels on both Xi,U and Xi,L. The model is hence expected to have the similar ECE on labeled and unlabeled data. Mako then performs temperature scaling (Platt, 1999) to minimize ECE on Xi,L, and then apply the same temperature on Xi,U to adjust the logits.
  After calibration, there could still exist data points labeled with low confidence. We then threshold Xi,U with a confidence of 1/nclasses + β for some small positive β. For example, in a 10-way classification problem, pick β = 0.01. We only keep the subset of Xi,U with calibrated confidence ≥ 0.1 + 0.01 = 0.11. Denote this kept portion of unlabeled data and labels as (Xi,U,c, Y πi,U,c).
sections.heading: 4.4 LML TASK TRAINING AND EVALUATION
sections.text: Finally, Xi,L, Yi,L, Xi,U,c and Y πi,U,c will be input into the mounted LML tool for the training of task Ti. The evaluation of the current model will be performed on all the hold-out testing data set (Xtest,1, Ytest,1), . . . , (Xtest,i, Ytest,i).
  Mako does not alter the internal procedures of the LML tool. Instead, it generates labels for unlabeled data and then feeds it to the LML tool as a black box. This design enables high modularity and the LML tool can be easily replaced. In the experiment section, we will demonstrate Mako’s performance when mounting on various of LML paradigms such as DF-CNN, DEN and TF. This design also imposes no additional knowledge space storage in LML.
sections.heading: 5 EXPERIMENTS
sections.heading: 5.1 DATASETS AND LML TASK SEQUENCES
sections.text: We evaluate Mako on LML task sequences generated from 3 commonly-used image classification datasets: MNIST (LeCun & Cortes, 2010), CIFAR-10 and CIFAR-100 (Krizhevsky, 2009).
  We mount Mako on existing supervised LML tools: DF-CNN, and TF (Lee et al., 2019; Bulat et al., 2020) to enable semi-supervised learning, and compare the performance to supervised LML on labeled data only as well as all data fully labeled. The task sequences are:
  1. Binary MNIST: for each task, pick classes 0 vs 1, 0 vs 2, ..., 8 vs 9, with 45 tasks in total. 2. Binary CIFAR-10: same construction as binary MNIST.
  Additionally, we compare Mako-mounted supervised LML tools to existing semi-supervised LML: CNNL, ORDisCo, and DistillMatch (Baucum et al., 2017; Wang et al., 2021; Smith et al., 2021). We generate the same task sequences as in their papers:
  3. 10-way CIFAR-10: the entire CIFAR-10 as one task. 4. 5-way CIFAR-100: for each task, pick each superclass in CIFAR-100, with 20 tasks in total. 5. 5-way MNIST: for each task, pick classes 0-4 and 5-9 in MNIST, with 2 tasks in total.
  Details of the tasks such as data splits, hyperparameters searched and labeling accuracy are explained in Appendix B.
sections.heading: 5.2 COMPARISON TO SUPERVISED LML
sections.text: For each task, data is split into a labeled training set, an unlabeled training set and a hold-out test set. The details of data split and weak labeler models searched are included in Appendix B.
  Table 1 and 2 summarize the performance of LML methods according to the amount of training data: L standing for the labeled training set and U with number denoting the number of instances in the unlabeled training set used during training of each task. These tables show peak per-task accuracy, final accuracy and catastrophic forgetting ratio up to the last task (Eqn. 6, 7 and 8, respectively). Additionally, we trained the LML methods on the same data with true labels instead of the Makogenerated labels to quantify the quality of the generated labels. LML methods achieve better peak
  per-task accuracy as more Mako-labeled data is provided for training. This shows that Mako is able to generate useful labels for training while the current task, and especially data distribution, keeps changing in the lifelong learning setting. Training with Mako labels has no significant improvement in final accuracy, but, compared to LML models trained on the true data, both peak per-task accuracy and final accuracy are at least 97% of the counterparts. Learning curves for each experiment are shown in Appendix C.
sections.heading: 5.3 COMPARISON TO SEMI-SUPERVISED LML
sections.text: As discussed in Section 2, there is little prior work in semi-supervised LML settings. We identify three approaches for comparison: CNNL, ORDisCo, and DistillMatch (Baucum et al., 2017; Wang et al., 2021; Smith et al., 2021). Each baseline approach contains multiple modules and networks which each require their own tuning; to enable fair comparisons, we replicate the experimental conditions (data set, amount of labeled vs. unlabeled data, task definitions, and network architecture) and compare our approach to the best results reported in each baseline’s original publication. Specifically, we replicate the instance-incremental learning experiments of CNNL on MNIST and CIFAR10, and the class-incremental learning experiments of ORDisCo and DistillMatch on CIFAR-10 and CIFAR-100, respectively. We briefly describe each experiment below; for more details, see Appendix B for data splits and the original publications for full experimental descriptions.
  Instance-Incremental Experiments. We compare semi-supervised learning using Mako in an instance-incremental setting (i.e., where all tasks are present at every epoch, but subsequent epochs contain different batches of unlabeled data), using an identical two-convolutional-layer CNN as described in (Baucum et al., 2017). The MNIST experiment uses a labeled data set of 150 images, with 1000 unlabeled images introduced in each of 30 instance-incremental batches. The CIFAR10 experiment uses a labeled data set of 2000 samples, with subsequent batches of 1000 unlabeled images. Each Mako experiment was run over 10 random seeds. For additional context, we include the performance of the same network trained in the instance-incremental setting where all data is labeled with the ground truth, representing an upper bound on semi-supervised performance. As shown in Table 3, using Mako labels in place of CNNL’s semi-supervised labeler results in comparable classification accuracy with better sample efficiency (MNIST), or strictly higher classification accuracy (CIFAR-10). Learning curves for each experiment are shown in Appendix D.
  Class-Incremental Experiments. We compare state-of-the-art LML methods (DF-CNN, DEN, and TF (Lee et al., 2019; Yoon et al., 2018; Bulat et al., 2020)) using Mako labels to perform semisupervised learning in a class-incremental setting (i.e., the model is sequentially presented with tasks containing new sets of classes). For all experiments below, each Mako-enabled approach was run
  over 10 random seeds, and we also include the performance of the same using all ground truth labels. We first compare Mako to ORDisCo over five binary CIFAR-10 tasks using 400 labeled instances, with the remaining data unlabeled. We note that we could not directly replicate the ORDisCo classification network architecture due to a lack of details in the original publication, but instead show that using Mako labels achieve a higher total classification accuracy with DF-CNN and TF using a significantly smaller network (4 convolutional layers for the Mako approaches, compared to ORDisCo’s 9 convolutional layers), shown in Table 4. Additionally, we compare Mako to DistillMatch over the 5-way CIFAR-100 using 20% labeled data, replicating DistillMatch’s ParentClass task. As shown in Table 4, all of the LML methods enabled by Mako labeling meet or exceed DistillMatch’s performance over the learning task. Learning curves for each experiment are shown in Appendix D.
sections.heading: 6 CONCLUSION AND FUTURE WORK
sections.text: In this paper, we identified the challenge that collecting task-level labeled data for LML is expensive. We address this challenge with data programming, where labels are automatically generated, and implemented the Mako framework that can be mounted on top of existing LML tools. Mako takes in a limited number of labeled data and an unlimited number of unlabeled data, aiming to minimize the performance gap to using the same data but fully labeled. We demonstrated Mako can achieve sufficiently high accuracy per task as well as resistance to catastrophic forgetting, while costing no additional knowledge base storage, over a set of common image classification LML task sequences.
  Future work on this topic can consider how to better resolve the issue of expensive labels at individual LML tasks. This includes how to extend Mako to a larger variety of LML task sequences. For instance, what alternative methods can be used for labeler hyperparameter tuning other than searching, what prior knowledge of tasks could assist automatic labeling, and how should the efficiencycapability trade-off of different models be balanced. Outside of the Mako framework, alternative solutions can modify existing LML tools directly, compromising a certain degree of modularity while possibly improving the overall LML performance.
  Another interesting future direction is to consider the scenario where the labeled data and unlabeled data input per task are drawn from different distributions. This can be enabled by optimal transport theory, specifically the minimization of Sinkhorn distance between two distributions (Cuturi, 2013). Ideally, this approach could allow unlabeled data from other datasets to assist LML, resolving the issue of not only expensive labels but also expensive data collection.
sections.heading: 7 REPRODUCIBILITY STATEMENT
sections.text: We encourage researchers to replicate our experiments. As discussed in Section 4, we have provided the step-by-step explanation of Mako workflow. We have included more experiment details in Appendix B for convenience. Additionally, we pushed a sample code to an anonymous GitHub repository (https://github.com/mako-anon/mako). All readers are welcomed to pull our code and execute Mako on the example data themselves.
sections.heading: APPENDIX A SNUBA THEORETICAL GUARANTEE AND PROOF
sections.text: Proposition 1 in Section 4.2 and its proof are given by the original Snuba paper (Varma & Ré, 2016). We adapt their notations to our paper to support the propositions in Section 4.2 on an arbitrary LML task Ti. For convenience, we remove the subscript i in all notations.
  Proof of Proposition 1: First, the probability of the l∞ error between the labeling accuracies of the generative model on XU and XL is greater than or equal to γ can be bounded by the following triangular inequality, with the empirical accuracy of one weak labeler in the middle.
  Pr[||aU,g − aL,g||∞ ≥ γ] ≤ Pr[||aU,g − aL,w||∞ + ||aL,w − aL,g||∞ ≥ γ] ≤ Pr[||aU,g − aL,w||∞ + ≥ γ]
  (11)
  Then, since we have h weak labelers, the following union bound holds for an individual labeler j in the committed set and its scalar accuracy.
  Pr[||aU,g − aL,w||∞ + ≥ γ] ≤ h∑ j=1 Pr[|aU,g,j − aL,w,j |+ ≥ γ] (12)
  In XL, suppose there are dj data points not labeled as abstain by labeler j, the definition of the empirical accuracy of weak labeler j is
  aL,w,j = 1
  dj ∑ k 1(yk = ŷj,k) (13)
  where yk is the ground truth label of data point k in the non-abstain subset and ˆyj,k is the label given by heuristic j. By combining Equations 12 and 13, and using Hoeffding’s inequality on the independent data points, we can get
  Pr[||aU,g − aL,w||∞ ≥ γ] = Pr[||aU,g − aL,w||∞+ ≥ γ − ]
  ≤ h∑ j=1 Pr[|aU,g,j − aL,w,j | ≥ γ − ]
  = h∑ j=1 Pr [ |aU,g,j − 1 dj ∑ k 1(yk = ŷj,k|) ≥ γ − ] ≤ h∑ j=1 2 exp(−2(γ − )2dj)
  ≤ 2h exp(−2(γ − )2 min(d1, . . . , dh))
  (14)
  Equation 14 bounds the probability of Snuba’s failure to keep ||aU,g − aL,g||∞ < γ in one iteration. Assume without loss of generality that we commit one more weak labeler per iteration, so for the all h iterations, we apply union bound again:
  Pr[||aU,g − aL,w||∞+ ≥ γ − for any iteration] ≤ h∑ j=1 Pr[||aU,g − aL,w||∞+ ≥ γ − for one iteration]
  ≤ 2h2 exp(−2(γ − )2 min(d1, . . . , dh))
  (15)
  Therefore, if we denote this probability of failure in any iteration as δ, we have the bound δ ≤ 2h2 exp(−2(γ − )2 min(d1, . . . , dh)) (16) and thus obtain Equation 9 by letting d = min(d1, . . . , dh).
sections.heading: APPENDIX B WEAK LABELER MODEL HYPERPARAMETERS IN
sections.text: EXPERIMENTS
  The LML task configurations are listed in Table 5 for the convenience of replicating our procedure. Because even after hyperparameter search (Section 4.1), there are still differences in the architectures and training hyperparameters throughout a single LML sequence, we have provided shrunken search spaces in weak labeler architecture and training hyperparameters.
  For example, to re-create our binary MNIST LML tasks, one needs to first arrange the 45 tasks as 0 vs 1, 0 vs 2, ..., 8 vs 9. Then, split the labeled training data, unlabeled training data and testing
  data into 120/11880/2000. All the data splits have balanced classes. Finally, perform our procedure described in Section 4, starting from the hyperparameter search in the shrunken search space.
  Figure 2 illustrates the searched architectures. The search spaces were inspired by various previous works on CNN designs (Lecun et al., 1998; He et al., 2016). Nonetheless, the final architectures are much smaller since we need fast training of multiple weak labelers. The notations of the figures are: out c: output channels/number of filters, k: size (width and height) of a filter, p: padding and s: stride.
  Figure 3 shows the labeling accuracies of the Mako-labeled data input into LML.
sections.heading: APPENDIX C ADDITIONAL SUPERVISED LML EXPERIMENT ANALYSIS
sections.text: We visualize peak per-task accuracy and final accuracy of supervised LML experiments (Figure 4) and learning curve of final accuracies in these experiments (Figure 5), summarized in Table 1 and 2. In these plots, we used the same notation L and U of the main text to specify the training data setting, except TrueU denoting the case trained on the same instances of U but using the true labels instead.
sections.heading: APPENDIX D ADDITIONAL SEMI-SUPERVISED LML EXPERIMENT
sections.text: ANALYSIS
  We include learning curves for each of the instance-incremental semi-supervised LML experiments in Figure 6 and each of the class-incremental semi-supervised LML experiments in Figure 7. All training curves show final accuracy up to task i as defined in Equation 3, where task i is the current task being trained. For easier tasks with high labeling accuracy, such as CIFAR-10, the Makoenabled semi-supervised LML methods perform similarly to the equivalent supervised fully-labeled task sequence. Otherwise, the Mako-enabled semi-supervised methods approach fully supervised performance.
  We additionally analyze the class-incremental semi-supervised LML baseline experiments for catastrophic forgetting, using the catastrophic forgetting ratio defined in Equation 4. Catastrophic forgetting measures are not available for the baseline methods ORDisCo and DistillMatch as they were not reported in the original publications. The results are shown in Table 6.
  For a further breakdown of results, we include task-specific performance measures for all of the semi-supervised LML experiments in Tables 7, 8, 9, and 10.
title: MAKO: SEMI-SUPERVISED CONTINUAL LEARNING WITH MINIMAL LABELED DATA VIA DATA PROGRAM- MING