Training Group Orthogonal Neural Networks with Privileged Information
Learning rich and diverse feature representation are always desired for deep convolutional neural networks (CNNs). Besides, when auxiliary annotations are available for specific data, simply ignoring them would be a great waste. In this paper, we incorporate these auxiliary annotations as privileged information and propose a novel CNN model that is able to maximize inherent diversity of a CNN model such that the model can learn better feature representation with a stronger generalization ability. More specifically, we propose a group orthogonal convolutional neural network (GoCNN) to learn features from foreground and background in an orthogonal way by exploiting privileged information for optimization, which automatically emphasizes feature diversity within a single model. Experiments on two benchmark datasets, ImageNet and PASCAL VOC, well demonstrate the effectiveness and high generalization ability of our proposed GoCNN models.
This paper proposes to learn groups of orthogonal features in a convnet by penalizing correlation among features in each group. The technique is applied in the setting of image classification with “privileged information” in the form of foreground segmentation masks, where the model is trained to learn orthogonal groups of foreground and background features using the correlation penalty and an additional “background suppression” term. Pros: Proposes a “group-wise model diversity” loss term which is novel, to my knowledge. The use of foreground segmentation masks to improve image classification is also novel. The method is evaluated on two standard and relatively large-scale vision datasets: ImageNet and PASCAL VOC 2012. Cons: The evaluation is lacking. There should be a baseline that leaves out the background suppression term, so readers know how much that term is contributing to the performance vs. the group orthogonal term. The use of the background suppression term is also confusing to me -- it seems redundant, as the group orthogonality term should already serve to suppress the use of background features by the foreground feature extractor. It would be nice to see the results with “Incomplete Privileged Information” on the full ImageNet dataset (rather than just 10% of it) with the privileged information included for the 10% of images where it’s available. This would verify that the method and use of segmentation masks remains useful even in the regime of more labeled classification data. The presentation overall is a bit confusing and difficult to follow, for me. For example, Section 4.2 is titled “A Unified Architecture: GoCNN”, yet it is not an overview of the method as a whole, but a list of specific implementation details (even the very first sentence). Minor: calling eq 3 a “regression loss” and writing “||0 - x||” rather than just “||x||” is not necessary and makes understanding more difficult -- I’ve never seen a norm regularization term written this way or described as a “regression to 0”. Minor: in fig. 1 I think the FG and BG suppression labels are swapped: e.g., the “suppress foreground” mask has 1s in the FG and 0s in the BG (which would suppress the BG, not the FG). An additional question: why are the results in Table 4 with 100% privileged information different from those in Table 1-2? Are these not the same setting? The ideas presented in this paper are novel and show some promise, but are currently not sufficiently ablated for readers to understand what aspects of the method are important. Besides additional experiments, the paper could also use some reorganization and revision for clarity. =============== Edit (1/29/17): after considering the latest revisions -- particularly the full ImageNet evaluation results reported in Table 5 demonstrating that the background segmentation *privileged information* is beneficial even with the full labeled ImageNet dataset -- I*ve upgraded my rating from 4 to 6. (I*ll reiterate a very minor point about Figure 1 though: I still think the *0* and *1* labels in the top part of the figures should be swapped to match the other labels. e.g., the topmost path in figure 1a, with the text *suppress foreground*, currently has 0 in the background and 1 in the foreground, when one would want the reverse of this to suppress the foreground.)
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Training Group Orthogonal Neural Networks with Privileged Information
Learning rich and diverse feature representation are always desired for deep convolutional neural networks (CNNs). Besides, when auxiliary annotations are available for specific data, simply ignoring them would be a great waste. In this paper, we incorporate these auxiliary annotations as privileged information and propose a novel CNN model that is able to maximize inherent diversity of a CNN model such that the model can learn better feature representation with a stronger generalization ability. More specifically, we propose a group orthogonal convolutional neural network (GoCNN) to learn features from foreground and background in an orthogonal way by exploiting privileged information for optimization, which automatically emphasizes feature diversity within a single model. Experiments on two benchmark datasets, ImageNet and PASCAL VOC, well demonstrate the effectiveness and high generalization ability of our proposed GoCNN models.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



This paper proposes to learn groups of orthogonal features in a convnet by penalizing correlation among features in each group. The technique is applied in the setting of image classification with “privileged information” in the form of foreground segmentation masks, where the model is trained to learn orthogonal groups of foreground and background features using the correlation penalty and an additional “background suppression” term. Pros: Proposes a “group-wise model diversity” loss term which is novel, to my knowledge. The use of foreground segmentation masks to improve image classification is also novel. The method is evaluated on two standard and relatively large-scale vision datasets: ImageNet and PASCAL VOC 2012. Cons: The evaluation is lacking. There should be a baseline that leaves out the background suppression term, so readers know how much that term is contributing to the performance vs. the group orthogonal term. The use of the background suppression term is also confusing to me -- it seems redundant, as the group orthogonality term should already serve to suppress the use of background features by the foreground feature extractor. It would be nice to see the results with “Incomplete Privileged Information” on the full ImageNet dataset (rather than just 10% of it) with the privileged information included for the 10% of images where it’s available. This would verify that the method and use of segmentation masks remains useful even in the regime of more labeled classification data. The presentation overall is a bit confusing and difficult to follow, for me. For example, Section 4.2 is titled “A Unified Architecture: GoCNN”, yet it is not an overview of the method as a whole, but a list of specific implementation details (even the very first sentence). Minor: calling eq 3 a “regression loss” and writing “||0 - x||” rather than just “||x||” is not necessary and makes understanding more difficult -- I’ve never seen a norm regularization term written this way or described as a “regression to 0”. Minor: in fig. 1 I think the FG and BG suppression labels are swapped: e.g., the “suppress foreground” mask has 1s in the FG and 0s in the BG (which would suppress the BG, not the FG). An additional question: why are the results in Table 4 with 100% privileged information different from those in Table 1-2? Are these not the same setting? The ideas presented in this paper are novel and show some promise, but are currently not sufficiently ablated for readers to understand what aspects of the method are important. Besides additional experiments, the paper could also use some reorganization and revision for clarity. =============== Edit (1/29/17): after considering the latest revisions -- particularly the full ImageNet evaluation results reported in Table 5 demonstrating that the background segmentation *privileged information* is beneficial even with the full labeled ImageNet dataset -- I*ve upgraded my rating from 4 to 6. (I*ll reiterate a very minor point about Figure 1 though: I still think the *0* and *1* labels in the top part of the figures should be swapped to match the other labels. e.g., the topmost path in figure 1a, with the text *suppress foreground*, currently has 0 in the background and 1 in the foreground, when one would want the reverse of this to suppress the foreground.)
assistant
This paper proposes to learn groups of orthogonal features in a convnet by penalizing correlation among features in each group. The technique is applied in the setting of image classification with “privileged information” in the form of foreground segmentation masks, where the model is trained to learn orthogonal groups of foreground and background features using the correlation penalty and an additional “background suppression” term. Pros: Proposes a “group-wise model diversity” loss term which is novel, to my knowledge. The use of foreground segmentation masks to improve image classification is also novel. The method is evaluated on two standard and relatively large-scale vision datasets: ImageNet and PASCAL VOC 2012. Cons: The evaluation is lacking. There should be a baseline that leaves out the background suppression term, so readers know how much that term is contributing to the performance vs. the group orthogonal term. The use of the background suppression term is also confusing to me -- it seems redundant, as the group orthogonality term should already serve to suppress the use of background features by the foreground feature extractor. It would be nice to see the results with “Incomplete Privileged Information” on the full ImageNet dataset (rather than just 10% of it) with the privileged information included for the 10% of images where it’s available. This would verify that the method and use of segmentation masks remains useful even in the regime of more labeled classification data. The presentation overall is a bit confusing and difficult to follow, for me. For example, Section 4.2 is titled “A Unified Architecture: GoCNN”, yet it is not an overview of the method as a whole, but a list of specific implementation details (even the very first sentence). Minor: calling eq 3 a “regression loss” and writing “||0 - x||” rather than just “||x||” is not necessary and makes understanding more difficult -- I’ve never seen a norm regularization term written this way or described as a “regression to 0”. Minor: in fig. 1 I think the FG and BG suppression labels are swapped: e.g., the “suppress foreground” mask has 1s in the FG and 0s in the BG (which would suppress the BG, not the FG). An additional question: why are the results in Table 4 with 100% privileged information different from those in Table 1-2? Are these not the same setting? The ideas presented in this paper are novel and show some promise, but are currently not sufficiently ablated for readers to understand what aspects of the method are important. Besides additional experiments, the paper could also use some reorganization and revision for clarity. =============== Edit (1/29/17): after considering the latest revisions -- particularly the full ImageNet evaluation results reported in Table 5 demonstrating that the background segmentation *privileged information* is beneficial even with the full labeled ImageNet dataset -- I*ve upgraded my rating from 4 to 6. (I*ll reiterate a very minor point about Figure 1 though: I still think the *0* and *1* labels in the top part of the figures should be swapped to match the other labels. e.g., the topmost path in figure 1a, with the text *suppress foreground*, currently has 0 in the background and 1 in the foreground, when one would want the reverse of this to suppress the foreground.)