Feature Intertwiner for Object Detection
A well-trained model should classify objects with unanimous score for every category. This requires the high-level semantic features should be alike among samples, despite a wide span in resolution, texture, deformation, etc. Previous works focus on re-designing the loss function or proposing new regularization constraints on the loss. In this paper, we address this problem via a new perspective. For each category, it is assumed that there are two sets in the feature space: one with more reliable information and the other with less reliable source. We argue that the reliable set could guide the feature learning of the less reliable set during training - in spirit of student mimicking teacher’s behavior and thus pushing towards a more compact class centroid in the high-dimensional space. Such a scheme also benefits the reliable set since samples become more closer within the same category - implying that it is easilier for the classifier to identify. We refer to this mutual learning process as feature intertwiner and embed the spirit into object detection. It is well-known that objects of low resolution are more difficult to detect due to the loss of detailed information during network forward pass. We thus regard objects of high resolution as the reliable set and objects of low resolution as the less reliable set. Specifically, an intertwiner is achieved by minimizing the distribution divergence between two sets. We design a historical buffer to represent all previous samples in the reliable set and utilize them to guide the feature learning of the less reliable set. The design of obtaining an effective feature representation for the reliable set is further investigated, where we introduce the optimal transport (OT) algorithm into the framework. Samples in the less reliable set are better aligned with the reliable set with aid of OT metric. Incorporated with such a plug-and-play intertwiner, we achieve an evident improvement over previous state-of-the-arts on the COCO object detection benchmark.
 This paper proposes a novel approach with the hypothesis that the reliable features can guide the less reliable ones. This approach is applied to the object detection task and show consistent performance improvements. pros) (+) This paper is well-written and easy to follow. (+) The base idea that divides the learned features into two sets; the reliable feature set and the less reliable one is very interesting and looks novel. Plus, the hypothesis, which is that reliable features can guide the features in the less reliable set is also interesting. (+) The performance improvements are quite large. (+) Extensive ablative studies are provided to support the proposed method well. cons) (-) The method of obtaining the representative in buffer B is not clearly presented. (-) The overall training and inference procedure are not clearly presented. (-) Some notations and descriptions are vague and confusing. (-) More than two datasets are necessary to show the effectiveness of the methods comments) - What is the higher level feature map P_m? and How did you choose the higher level feature map at the m-th level in option (b) and (c) in Section 3.3. - What is the meaning of the "past" features in Section 3.2? - It is better to show the exact architecture of the make-up module and the critic module. - Can this method apply to the other backbones such as VGG or ResNets without FPN? - The sentences at the bottom of p.4 starting with "Note that only~" looks ambiguous. - f_critic^j may be the j-th element of F_critic, please denote what f_critic^j stands for. Even if the paper needs to be revised for better readability, I think this paper is above the standard of ICLR because the idea is interesting and novel. Furthermore, the experimental studies are properly designed and well support the main idea. I am leaning toward acceptance, but I would like to see the other reviewers* comments.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Feature Intertwiner for Object Detection
A well-trained model should classify objects with unanimous score for every category. This requires the high-level semantic features should be alike among samples, despite a wide span in resolution, texture, deformation, etc. Previous works focus on re-designing the loss function or proposing new regularization constraints on the loss. In this paper, we address this problem via a new perspective. For each category, it is assumed that there are two sets in the feature space: one with more reliable information and the other with less reliable source. We argue that the reliable set could guide the feature learning of the less reliable set during training - in spirit of student mimicking teacher’s behavior and thus pushing towards a more compact class centroid in the high-dimensional space. Such a scheme also benefits the reliable set since samples become more closer within the same category - implying that it is easilier for the classifier to identify. We refer to this mutual learning process as feature intertwiner and embed the spirit into object detection. It is well-known that objects of low resolution are more difficult to detect due to the loss of detailed information during network forward pass. We thus regard objects of high resolution as the reliable set and objects of low resolution as the less reliable set. Specifically, an intertwiner is achieved by minimizing the distribution divergence between two sets. We design a historical buffer to represent all previous samples in the reliable set and utilize them to guide the feature learning of the less reliable set. The design of obtaining an effective feature representation for the reliable set is further investigated, where we introduce the optimal transport (OT) algorithm into the framework. Samples in the less reliable set are better aligned with the reliable set with aid of OT metric. Incorporated with such a plug-and-play intertwiner, we achieve an evident improvement over previous state-of-the-arts on the COCO object detection benchmark.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 This paper proposes a novel approach with the hypothesis that the reliable features can guide the less reliable ones. This approach is applied to the object detection task and show consistent performance improvements. pros) (+) This paper is well-written and easy to follow. (+) The base idea that divides the learned features into two sets; the reliable feature set and the less reliable one is very interesting and looks novel. Plus, the hypothesis, which is that reliable features can guide the features in the less reliable set is also interesting. (+) The performance improvements are quite large. (+) Extensive ablative studies are provided to support the proposed method well. cons) (-) The method of obtaining the representative in buffer B is not clearly presented. (-) The overall training and inference procedure are not clearly presented. (-) Some notations and descriptions are vague and confusing. (-) More than two datasets are necessary to show the effectiveness of the methods comments) - What is the higher level feature map P_m? and How did you choose the higher level feature map at the m-th level in option (b) and (c) in Section 3.3. - What is the meaning of the "past" features in Section 3.2? - It is better to show the exact architecture of the make-up module and the critic module. - Can this method apply to the other backbones such as VGG or ResNets without FPN? - The sentences at the bottom of p.4 starting with "Note that only~" looks ambiguous. - f_critic^j may be the j-th element of F_critic, please denote what f_critic^j stands for. Even if the paper needs to be revised for better readability, I think this paper is above the standard of ICLR because the idea is interesting and novel. Furthermore, the experimental studies are properly designed and well support the main idea. I am leaning toward acceptance, but I would like to see the other reviewers* comments.
assistant
 This paper proposes a novel approach with the hypothesis that the reliable features can guide the less reliable ones. This approach is applied to the object detection task and show consistent performance improvements. pros) (+) This paper is well-written and easy to follow. (+) The base idea that divides the learned features into two sets; the reliable feature set and the less reliable one is very interesting and looks novel. Plus, the hypothesis, which is that reliable features can guide the features in the less reliable set is also interesting. (+) The performance improvements are quite large. (+) Extensive ablative studies are provided to support the proposed method well. cons) (-) The method of obtaining the representative in buffer B is not clearly presented. (-) The overall training and inference procedure are not clearly presented. (-) Some notations and descriptions are vague and confusing. (-) More than two datasets are necessary to show the effectiveness of the methods comments) - What is the higher level feature map P_m? and How did you choose the higher level feature map at the m-th level in option (b) and (c) in Section 3.3. - What is the meaning of the "past" features in Section 3.2? - It is better to show the exact architecture of the make-up module and the critic module. - Can this method apply to the other backbones such as VGG or ResNets without FPN? - The sentences at the bottom of p.4 starting with "Note that only~" looks ambiguous. - f_critic^j may be the j-th element of F_critic, please denote what f_critic^j stands for. Even if the paper needs to be revised for better readability, I think this paper is above the standard of ICLR because the idea is interesting and novel. Furthermore, the experimental studies are properly designed and well support the main idea. I am leaning toward acceptance, but I would like to see the other reviewers* comments.