Few-shot Learning by Focusing on Differences
Few-shot classification may involve differentiating data that belongs to a different level of labels granularity. Compounded by the fact that the number of available labeled examples are scarce in the novel classification set, relying solely on the loss function to implicitly guide the classifier to separate data based on its label might not be enough; few-shot classifier needs to be very biased to perform well. In this paper, we propose a model that incorporates a simple prior: focusing on differences by building a dissimilar set of class representations. The model treats a class representation as a vector and removes its component that is shared among closely related class representatives. It does so through the combination of learned attention and vector orthogonalization. Our model works well on our newly introduced dataset, Hierarchical-CIFAR, that contains different level of labels granularity. It also substantially improved the performance on fine-grained classification dataset, CUB; whereas staying competitive on standard benchmarks such as mini-Imagenet, Omniglot, and few-shot dataset derived from CIFAR.
In this paper the authors propose a metric based model for few-shot learning. The goal of the proposed technique is to incorporate a prior that highlight better the dissimilarity between closely related class prototype. Thus, the proposed paper is related to prototypical neural network (use of prototype to represent a class) but differ from it by using inner product scoring as a similarity measure instead of the use of euclidean distance. There is also close similarity between the proposed method and matching network overall, the paper does not highlight the novelty of their proposed method especially prototypical network and matching network. Thus, the related work session is so general and does not tackle the close models in details. The experiments do not provide convincing evidence of the correctness of the proposed approach. Several parts are unclear/incomprehensible: (1) The Introduction is confusing and does not demonstrate the problem that the paper is trying to solve. Specifically, the described intuition (Mill’s method of difference) is not convincing (2) the first sentence of the section “Our work.” (page 1) is long and unclear … “In this paper, we propose a model that focuses on the differences in the support set of closely related classes in assigning the class label to a new instance in the novel task.” (3) the use of the two level of embedding is confusing and not clear. Figure 1 is also confusion and not clear. the correctness of the proposed approach is not proved by the conducted experiment and does not provide convincing and fair comparison with SoA techniques: (1) The experiments do not provide the details of the used architecture compared to your baseline. (how many layers are used in both embedding systems) (2) In Table 1 you are using the results reported by Chen et al. (2019) did you use his framework (Resnet or 4 layers CNN) **Minor comments** The definition of the embedding function f = (f_g o f_f) (in line 1 page 5) is not consistent with the domain of each function f_g is defined on R^H x R^H. K is not defined (last line page 3)
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Few-shot Learning by Focusing on Differences
Few-shot classification may involve differentiating data that belongs to a different level of labels granularity. Compounded by the fact that the number of available labeled examples are scarce in the novel classification set, relying solely on the loss function to implicitly guide the classifier to separate data based on its label might not be enough; few-shot classifier needs to be very biased to perform well. In this paper, we propose a model that incorporates a simple prior: focusing on differences by building a dissimilar set of class representations. The model treats a class representation as a vector and removes its component that is shared among closely related class representatives. It does so through the combination of learned attention and vector orthogonalization. Our model works well on our newly introduced dataset, Hierarchical-CIFAR, that contains different level of labels granularity. It also substantially improved the performance on fine-grained classification dataset, CUB; whereas staying competitive on standard benchmarks such as mini-Imagenet, Omniglot, and few-shot dataset derived from CIFAR.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



In this paper the authors propose a metric based model for few-shot learning. The goal of the proposed technique is to incorporate a prior that highlight better the dissimilarity between closely related class prototype. Thus, the proposed paper is related to prototypical neural network (use of prototype to represent a class) but differ from it by using inner product scoring as a similarity measure instead of the use of euclidean distance. There is also close similarity between the proposed method and matching network overall, the paper does not highlight the novelty of their proposed method especially prototypical network and matching network. Thus, the related work session is so general and does not tackle the close models in details. The experiments do not provide convincing evidence of the correctness of the proposed approach. Several parts are unclear/incomprehensible: (1) The Introduction is confusing and does not demonstrate the problem that the paper is trying to solve. Specifically, the described intuition (Mill’s method of difference) is not convincing (2) the first sentence of the section “Our work.” (page 1) is long and unclear … “In this paper, we propose a model that focuses on the differences in the support set of closely related classes in assigning the class label to a new instance in the novel task.” (3) the use of the two level of embedding is confusing and not clear. Figure 1 is also confusion and not clear. the correctness of the proposed approach is not proved by the conducted experiment and does not provide convincing and fair comparison with SoA techniques: (1) The experiments do not provide the details of the used architecture compared to your baseline. (how many layers are used in both embedding systems) (2) In Table 1 you are using the results reported by Chen et al. (2019) did you use his framework (Resnet or 4 layers CNN) **Minor comments** The definition of the embedding function f = (f_g o f_f) (in line 1 page 5) is not consistent with the domain of each function f_g is defined on R^H x R^H. K is not defined (last line page 3)
assistant
In this paper the authors propose a metric based model for few-shot learning. The goal of the proposed technique is to incorporate a prior that highlight better the dissimilarity between closely related class prototype. Thus, the proposed paper is related to prototypical neural network (use of prototype to represent a class) but differ from it by using inner product scoring as a similarity measure instead of the use of euclidean distance. There is also close similarity between the proposed method and matching network overall, the paper does not highlight the novelty of their proposed method especially prototypical network and matching network. Thus, the related work session is so general and does not tackle the close models in details. The experiments do not provide convincing evidence of the correctness of the proposed approach. Several parts are unclear/incomprehensible: (1) The Introduction is confusing and does not demonstrate the problem that the paper is trying to solve. Specifically, the described intuition (Mill’s method of difference) is not convincing (2) the first sentence of the section “Our work.” (page 1) is long and unclear … “In this paper, we propose a model that focuses on the differences in the support set of closely related classes in assigning the class label to a new instance in the novel task.” (3) the use of the two level of embedding is confusing and not clear. Figure 1 is also confusion and not clear. the correctness of the proposed approach is not proved by the conducted experiment and does not provide convincing and fair comparison with SoA techniques: (1) The experiments do not provide the details of the used architecture compared to your baseline. (how many layers are used in both embedding systems) (2) In Table 1 you are using the results reported by Chen et al. (2019) did you use his framework (Resnet or 4 layers CNN) **Minor comments** The definition of the embedding function f = (f_g o f_f) (in line 1 page 5) is not consistent with the domain of each function f_g is defined on R^H x R^H. K is not defined (last line page 3)