Meta-learning is a promising strategy for learning to efficiently learn within new tasks, using data gathered from a distribution of tasks. However, the meta-learning literature thus far has focused on the task segmented setting, where at train-time, offline data is assumed to be split according to the underlying task, and at test-time, the algorithms are optimized to learn in a single task. In this work, we enable the application of generic meta-learning algorithms to settings where this task segmentation is unavailable, such as continual online learning with a time-varying task. We present meta-learning via online changepoint analysis (MOCA), an approach which augments a meta-learning algorithm with a differentiable Bayesian changepoint detection scheme. The framework allows both training and testing directly on time series data without segmenting it into discrete tasks. We demonstrate the utility of this approach on a nonlinear meta-regression benchmark as well as two meta-image-classification benchmarks.
The paper proposes a solution for using meta-learning methods without the need of determining the task segmentation a priori. Authors identify that the task segmented setting is very similar to the problem of change-point detection (CPD) and particularly, they connect the generative model of the meta-learning approach to the Bayesian recursive method of Adams and MacKay 2007. The presentation of the meta-learning problem is well done and I understand its importance within the difficulties for modelling new unobserved tasks. The notation and description of the generative model included in the problem statement section is clearly understandable for any reader, this is a very positive point. They demonstrate a deep comprehension of the BOCPD model of Adams, and its extension for the meta-learning approach is original to me. Lastly, the presentation of the MOCA meta-learning algorithm is useful for reproducibility and I see it completely applicable to other scenarios. There is a noticeable effort of describing the solution for both regression and classification problems and the empirical results conclude a positive performance of MOCA.
Overall, I consider that the paper is well-written with thorough explanations. Of course, there are some details that could be improved (I will comment this later). If done, I would be willing to increase my score. The contribution of the paper is significant to meta-learning models and I think it will help to spread the Adams’ model to other type of problems.
There is an important contribution in the paper that I have to mention and it is also relevant for the future application of the Adams model. If one reads the original BOCPD paper, in particular Eq (1), where the predictive posterior p(x_t+1|x_{1:t}) is defined from a marginalisation over the run length values, it is noticeable that this equation is not used in the final recursion of the CPD method. This is because the posterior p(r_t|x_{1:t}) is sufficient for determining if there is a CP on the t time-step or not. So the predictive posterior is never used in practice (in the original paper). When I first read Adams 2007, this detail was clear to me. Surprisingly, I find that the authors have find a practical use of this equation (Eq. (7)) and it is in the main core of the meta-learning algorithm, this is fantastic.
The details I think should be improved are:
- The conditional run-length prior p(r_t|r_{t-1}) barely appears and without any detailed description is not obvious for non-familiar readers with the Bayesian CPD approach.
- The first time I read the manuscript, the use of z_t in the BOCPD presentation made me feel a bit lost. Why not use x or y? At least specify that is a toy variable for the explanation. Later on, authors change again to the x,y notation.
- Equations 3 and 4 are too similar, this looks a bit repetitive. Why not reusing one of them or say the change from one term to another? 
- If reading the literature of the BOCPD and posterior extensions, the likelihood model of the detector is often referred as the underlying predictive model (UPM). Using a similar term would help for orienting familiar readers into the solution.
- As authors should have noted on their experiments, as one makes the run-length higher, the number of parameters \eta[r_t] increases. A clear state on how this is solved would help.
- In the PCOC subsection, I find the definition of y \sim q(y) a bit weird. Using Cat(_) or Multinomial likelihood notation would be a bit better.