Conventional deep learning classifiers are static in the sense that they are trained on a predefined set of classes and learning to classify a novel class typically requires re-training. In this work, we address the problem of Low-Shot network-expansion learning. We introduce a learning framework which enables expanding a pre-trained (base) deep network to classify novel classes when the number of examples for the novel classes is particularly small. We present a simple yet powerful distillation method where the base network is augmented with additional weights to classify the novel classes, while keeping the weights of the base network unchanged. We term this learning hard distillation, since we preserve the response of the network on the old classes to be equal in both the base and the expanded network. We show that since only a small number of weights needs to be trained, the hard distillation excels for low-shot training scenarios. Furthermore, hard distillation avoids detriment to classification performance on the base classes. Finally, we show that low-shot network expansion can be done with a very small memory footprint by using a compact generative model of the base classes training data with only a negligible degradation relative to learning with the full training set.
The paper proposes a method for adapting a pre-trained network, trained on a fixed number of
classes, to incorporate novel classes for doing classification, especially when the novel classes
only have a few training examples available. They propose to do a `hard' distillation, i.e. they
introduce new nodes and parameters to the network to add the new classes, but only fine-tune the new
networks without modifying the original parameters. This ensures that, in the new expanded and
fine-tuned network, the class confusions will only be between the old and new classes and not
between the old classes, thus avoiding catastrophic forgetting. In addition they use GMMs trained on
the old classes during the fine-tuning process, thus avoiding saving all the original training data.
They show experiments on public benchmarks with three different scenarios, i.e.  base and novel
classes from different domains, base and novel classes from the same domain and novel classes have
similarities among themselves, and base and novel classes from the same domain and each novel class
has similarities with at least one of the base class.                        
                                                                             
- The paper is generally well written and it is clear what is being done     
- The idea is simple and novel; to the best of my knowledge it has not been tested before
- The method is compared with Nearest Class Means (NCM) and Prototype-kNN with soft distillation
  (iCARL; where all weights are fine-tuned). The proposed method performs better in low-shot
  settings and comparably when large number of training examples of the novel classes are available
- My main criticism will be the limited dataset size on which the method is validated. The ILSVRC12
  subset contains 5 base and 5 novel classes and the UT-Zappos50K subset also has 10 classes. The
  idea is simple and novel, which is good, but the validation is limited and far from any realistic
  use. Having only O(10) classes is not convincing, especially when the datasets used do have large
  number of classes. I agree that this will not allow or will takes some involved manual effort to
  curate subsets for the settings proposed, but it is necessary for being convincing.