A CLOSER LOOK AT FEW-SHOT CLASSIFICATION
Few-shot classification aims to learn a classifier to recognize unseen classes during training with limited labeled examples. While significant progress has been made, the growing complexity of network designs, meta-learning algorithms, and differences in implementation details make a fair comparison difficult. In this paper, we present 1) a consistent comparative analysis of several representative few-shot classification algorithms, with results showing that deeper backbones significantly reduce the performance differences among methods on datasets with limited domain differences, 2) a modified baseline method that surprisingly achieves competitive performance when compared with the state-of-the-art on both the miniImageNet and the CUB datasets, and 3) a new experimental setting for evaluating the cross-domain generalization ability for few-shot classification algorithms. Our results reveal that reducing intra-class variation is an important factor when the feature backbone is shallow, but not as critical when using deeper backbones. In a realistic cross-domain evaluation setting, we show that a baseline method with a standard fine-tuning practice compares favorably against other state-of-the-art few-shot learning algorithms.
This paper gives a nice overview of existing works on few-shot learning. It groups them into some intuitive categories and meanwhile distills a common framework (Figure 2) employed by the methods. Moreover, the authors selected four of them, along with two baselines, to experimentally compare their performances under a cleaned experiment protocol. 
The experiments cover three few-shot learning scenarios respectively for generic object recognition, fine-grained classification, and cross-domain adaptation. While I do *not* think the third scenario is “more practical”, it is certainly nice to have it included in the experiments. 
The experiment setup is unfortunately questionable. Since there is a validation set, one should use it to determine the free parameters (e.g., the number of epochs, learning rates, etc.). However, it seems like the same set of free parameters are used for different methods, making the comparison unfair because this set may favor some methods and yet hurt the others.  
The results of RelationNet are missing in Table 4.
Another concern is that the same number of novel classes is used in the training and the testing stage. A more practical application of the learned meta model is to use it to handle different testing scenarios. There could be five novel classes in one scenario, 10 novel classes in another, and 100 in the third, etc. The number of labeled examples per class may also vary from one testing scenario to anther. 
It is misleading by the following: “Very recently, Motiian et al. (2017) addresses the few-shot domain adaptation problem.” There are a few variations in domain adaptation (DA). The learner has access to the fully labeled source domain and a small set of labeled target examples in supervised DA, to the source domain, a couple of labeled target examples, and many unlabeled target examples in semi-supervised DA, and to the source domain and many unlabeled target data points in the unsupervised DA. These have been studied long before (Motiian et al., 2017), for instance the works of Saenko et al. (2010) and Gong et al. (2013). 
[ref] Saenko K, Kulis B, Fritz M, Darrell T. Adapting visual category models to new domains. InEuropean conference on computer vision 2010 Sep 5 (pp. 213-226). Springer, Berlin, Heidelberg.
[ref] Gong B, Grauman K, Sha F. Connecting the dots with landmarks: Discriminatively learning domain-invariant features for unsupervised domain adaptation. InInternational Conference on Machine Learning 2013 Feb 13 (pp. 222-230).
Overall, the paper is well written and may serve as a nice survey of existing works on few-shot learning. The unified experiment setup can facilitate the future research for fair comparisons, along with the three testing scenarios. However, I have some concerns as above about the experiment setups and hence also the conclusions.