Learning to Model the Tail
We describe an approach to learning from long-tailed, imbalanced datasets that are prevalent in real-world settings. Here, the challenge is to learn accurate "fewshot" models for classes in the tail of the class distribution, for which little data is available. We cast this problem as transfer learning, where knowledge from the data-rich classes in the head of the distribution is transferred to the data-poor classes in the tail. Our key insights are as follows. First, we propose to transfer meta-knowledge about learning-to-learn from the head classes. This knowledge is encoded with a meta-network that operates on the space of model parameters, that is trained to predict many-shot model parameters from few-shot model parameters. Second, we transfer this meta-knowledge in a progressive manner, from classes in the head to the "body", and from the "body" to the tail. That is, we transfer knowledge in a gradual fashion, regularizing meta-networks for few-shot regression with those trained with more training data. This allows our final network to capture a notion of model dynamics, that predicts how model parameters are likely to change as more training data is gradually added. We demonstrate results on image classification datasets (SUN, Places, and ImageNet) tuned for the long-tailed setting, that significantly outperform common heuristics, such as data resampling or reweighting.
Summary
	-------
	The paper proposes an approach for transfer learning for
	multi-class classification problems that aids the learning of categories
	with few training examples (the categories in the tail of the
	distribution of numbers of examples per category). It is based
	on ideas of meta-learning: it builds a (meta-)model of the dynamics
	that accompany the change in model parameters as more training
	data is made available to a classifier.
	Specifically, the proposed approach takes inspiration from
	existing work on meta-learning [20] but extends it by applying
	it to CNNs, utilizing deep residual networks as the
	meta-model, and applying the framework to a general 'long-tail
	problem' setting in which the number of training examples
	available is different and not fixed between categories.
	Experiments are conducted on curated versions of existing
	datasets (curated such that they exhibit strong long-tail
	distributions): SUN-397 [13], Places [7], and ImageNet
	[5]. The performance of the proposed method is demonstrated to
	be considerably higher than several more adhoc baselines from
	the literature.
	Novelty and significance
	------------------------
	While the proposed method draws inspiration from a variety of
	existing methods in transfer learning (notably [20]), the
	particular formulation using deep residual networks as a
	meta-model, modeling the dynamics of adding progressively more
	training examples, and handling the case of different numbers
	of available training examples per category in a unified and
	automatic way are novel aspects of the paper to my knowledge.
	These aspects could potentially be useful to a wider range of
	applications than (image) classification and facilitate the
	treatment of long-tail problems, as they arise in many
	real-world application scenarios.
	Technical correctness
	---------------------
	The proposed method of (meta-)modeling the gradual addition of
	training examples seems plausible and technically correct.
	It would, however, be interesting to learn more about the
	complexity of the proposed approach in terms of training time,
	since a significant number of individual models has to be
	trained prior to training the meta-model.
	Experimental evalution
	----------------------
	The experimental evaluation seems convincing: it is done on
	three different existing datasets, compares the proposed
	approach to existing, more ad-hoc baselines as well as [20],
	and highlights the contributions to performance of the
	different proposed system components (notably, taking the step
	from fixed to recursive splitting of categories). Performance
	is indeed best for the proposed method, but considerable
	margins, and particularly pronounced for tail categories.
	Presentation
	------------
	The paper text seems to suffer from a latex compilation issue
	that results in missing multiple bibliographic and figure
	references (questions marks instead of numbers: line 79, 176,
	239, 277, Tab. 1), which is unfortunate and a possible
	indication that the paper submission has been rushed or a
	premature version uploaded. The text, hoever, does contain
	enough information in order to know what is going on despite
	not having the references.
	The authors should please comment on this and what the correct
	references are in their rebuttal.
	Apart from this issue, the paper is well written and clearly
	presents the proposed ideas, experimental results, and
	relation to prior work.
	Typo line 74: 'changeling' - challenging; line 96: 'the the'