FEW-SHOT LEARNING WITH GRAPH NEURAL NET- WORKS
We propose to study the problem of few-shot learning with the prism of inference on a partially observed graphical model, constructed from a collection of input images whose label can be either observed or not. By assimilating generic message-passing inference algorithms with their neural-network counterparts, we define a graph neural network architecture that generalizes several of the recently proposed few-shot learning models. Besides providing improved numerical performance, our framework is easily extended to variants of few-shot learning, such as semi-supervised or active learning, demonstrating the ability of graph-based models to operate well on 'relational' tasks.
This paper introduces a graph neural net approach to few-shot learning. Input examples form the nodes of the graph and edge weights are computed as a nonlinear function of the absolute difference between node features. In addition to standard supervised few-shot classification, both semi-supervised and active learning task variants are introduced. The proposed approach captures several popular few-shot learning approaches as special cases. Experiments are conducted on both Omniglot and miniImagenet datasets.
Strengths
- Use of graph neural nets for few-shot learning is novel.
- Introduces novel semi-supervised and active learning variants of few-shot classification.
Weaknesses
- Improvement in accuracy is small relative to previous work.
- Writing seems to be rushed.
The originality of applying graph neural networks to the problem of few-shot learning and proposing semi-supervised and active learning variants of the task are the primary strengths of this paper. Graph neural nets seem to be a more natural way of representing sets of items, as opposed to previous approaches that rely on a random ordering of the labeled set, such as the FCE variant of Matching Networks or TCML. Others will likely leverage graph neural net ideas to further tackle few-shot learning problems in the future, and this paper represents a first step in that direction.
Regarding the graph, I am wondering if the authors can comment on what scenarios is the graph structure expected to help? In the case of 1-shot, the graph can only propagate information about other classes, which seems to not be very useful.
Though novel, the motivation behind the semi-supervised and active learning setup could use some elaboration. By including unlabeled examples in an episode, it is already known that they belong to one of the K classes. How realistic is this set-up and in what application is it expected that this will show up?
For active learning, the proposed method seems to be specific to the case of obtaining a single label. How can the proposed method be scaled to handle multiple requested labels?
Overall the paper is well-structured and related work covers the relevant papers, but the details of the paper seem hastily written.
In the problem set-up section, it is not immediately clear what the distinction between s, r, and t is. Stating more explicitly that s is for the labeled data, etc. would make this section easier to follow. In addition, I would suggest stating the reason why t=1 is a necessary assumption for the proposed model in the few-shot and semi-supervised cases.
Regarding the Omniglot dataset, Vinyals et al. (2016) augmented the classes so that 4,800 classes were used for training and 1,692 for test. Was the same procedure done for the experiments in the paper? If yes, please update 6.1.1 to make this distinction more clear. If not, please update the experiments to be consistent with the baselines.
In the experiments, does the \varphi MLP explicitly enforce symmetry and identity or is it learned?
Regarding the Omniglot baselines, it appears that Koch et al. (2015), Edwards & Storkey (2016), and Finn et al. (2017) use non-standard class splits relative to the other methods. This should probably be noted.
The results for Prototypical Networks appear to be incorrect in the Omniglot and Mini-Imagenet tables. According to Snell et al. (2017) they should be 49.4% and 68.2% for miniImagenet. Moreover, Snell et al. (2017) only used 64 classes for training instead of 80 as utilized in the proposed approach. Given this, I am wondering if the authors can comment on the performance difference in the 5-shot case, even though Prototypical Networks is a special case of GNNs?
For semi-supervised and active-learning results, please include error bars for the miniImagenet results. Also, it would be interesting to see 20-way results for Omniglot as the gap between the proposed method and the baseline would potentially be wider.
Other Comments:
- In Section 4.2, Gc(.) is defined in Equation 2 but not mentioned in the text.
- In Section 4.3, adding an equation to clarify the relationship with Matching Networks would be helpful.
- I believe there is a typo in section 4.3 in that softmax(\varphi) should be softmax(-\varphi), so that more similar pairs will be more heavily weighted.
- The equation in 5.1 appears to be missing a minus sign.
Overall, the paper is novel and interesting, though the clarity and experimental results could be better explained.
EDIT: I have read the author's response. The writing is improved and my concerns have largely been addressed. I am therefore revising my rating of the paper to a 7.