We study many-class few-shot (MCFS) problem in both supervised learning and meta-learning scenarios. Compared to the well-studied many-class many-shot and few-class few-shot problems, MCFS problem commonly occurs in practical applications but is rarely studied. MCFS brings new challenges because it needs to distinguish between many classes, but only a few samples per class are available for training. In this paper, we propose "memory-augmented hierarchical-classification network (MahiNet)" for MCFS learning. It addresses the "many-class" problem by exploring the class hierarchy, e.g., the coarse-class label that covers a subset of fine classes, which helps to narrow down the candidates for the fine class and is cheaper to obtain. MahiNet uses a convolutional neural network (CNN) to extract features, and integrates a memory-augmented attention module with a multi-layer perceptron (MLP) to produce the probabilities over coarse and fine classes. While the MLP extends the linear classifier, the attention module extends a KNN classifier, both together targeting the "few-shot" problem. We design different training strategies of MahiNet for supervised learning and meta-learning. Moreover, we propose two novel benchmark datasets "mcfsImageNet" (as a subset of ImageNet) and "mcfsOmniglot" (re-splitted Omniglot) specifically for MCFS problem. In experiments, we show that MahiNet outperforms several state-of-the-art models on MCFS classification tasks in both supervised learning and meta-learning scenarios.
This paper presents methods for (1) adding inductive bias to a classifier through coarse-to-fine prediction along a class hierarchy and (2) learning a memory-based KNN classifier through an intuitive procedure that keeps track of mislabeled instances during learning. Further, the paper motivates focused work on the many class / few shot classification scenario and creates new benchmark datasets from subsets of imagenet and omniglot that match this scenario. Experimental results show gains over popular competing methods on these benchmarks.
Overall, I like the motivation that this paper provides for many class / few shot and find some of the methods proposed interesting. Yet there are issues with clarity of presentation that made it somewhat difficult to fully understand the exact procedures that were implemented. The model figure is useful, but could be refined to add additional clarity -- particularly in the case of the KNN learning procedure. 
I'm not entirely familiar with recent work in this sub-field, so it is difficult for me to judge the novelty of the proposed procedure. Is it really true that class-hierarchies have never been used to perform coarse-to-fine inference in past work? If so, this should be state clearly. If not, related work should be mentioned and compared against. Finally, while the procedures are intuitive -- the takeaway of this paper could be substantially improved if even simple theoretical analysis were provided. For example, in the limit of infinite data, does the memory-based KNN learning procedure actually produce the right classifier? 
Misc comments questions:
-The paper says at least twice that coarse classification will be performed with an MLP, while fine classification will use a KNN -- yet, the model section also state that both coarse and fine use both MLP and KNN. It is unclear to me which model setup was used in experiments. 
-Can anything theoretical be shown about the class hierarchy based classification technique? Intuitively, it does add inductive bias through a manually defined taxonomy, but can something more precise be said about how it restricts the hypothesis space? This procedure is simple enough that I would be surprised if similar techniques had not be studied thoroughly in the statistical learning theory literature. 
-The procedure for updating the KNN memory is intuitive, but can anything more be said about it? In isolation, is the KNN learning procedure at least consistent -- i.e. in the limit of large data does it converge to the correct classifier? Maybe this is trivial to prove, but is worth including.