Decoupling Representation and Classifier for Long-Tailed Recognition
The long-tail distribution of the visual world poses great challenges for deep learning based classification models on how to handle the class imbalance problem. Existing solutions usually involve class-balancing strategies, e.g., by loss re-weighting, data re-sampling, or transfer learning from head- to tail-classes, but all of them adhere to the scheme of jointly learning representations and classifiers. In this work, we decouple the learning procedure into representation learning and classification, and systematically explore how different balancing strategies affect them for long-tailed recognition. The findings are surprising: (1) data imbalance might not be an issue in learning high-quality representations; (2) with representations learned with the simplest instance-balanced (natural) sampling, it is also possible to achieve strong long-tailed recognition ability at little to no cost by adjusting only the classifier. We conduct extensive experiments and set new state-of-the-art performance on common long-tailed benchmarks like ImageNet-LT, Places-LT and iNaturalist, showing that it is possible to outperform carefully designed losses, sampling strategies, even complex modules with memory, by using a straightforward approach that decouples representation and classification.
The paper tries to handle the class imbalance problem by decoupling the learning process into representation learning and classification, in contrast to the current methods that jointly learn both of them. They comprehensively study several sampling methods for representation learning and different strategies for classification. They find that instance-balanced sampling gives the best representation, and simply adjusting the classifier will equip the model with long-tailed recognition ability. They achieve start of art on long-tailed data (ImageNet-LT, Places-LT and iNaturalist). In general, this is paper is an interesting paper. The author propose that instance-balanced sampling already learns the best and most generalizable representations, which is out of common expectation. They perform extensive experiment to illustrate their points. --Writing: This paper is well written in English and is well structured. And there are two typos. One is in the second row of page 3, *… a more continuous decrease [in in] class labels …* and the other one is in the first paragraph of section 5.4, *… report state-of-art results [on on] three common long-tailed benchmarks …*. --Introduction and review: The authors do a comprehensive literature review, listing the main directions for solving the long-tailed recognition problem. They emphasis that these methods all jointly learn the representations and classifiers, which *make it unclear how the long-tailed recognition ability is achieved-is it from learning a better representation or by handling the data imbalance better via shifting classifier decision boundaries*. This motivate them to decouple representation learning and classification. --Experiment: Since this paper decouples the representation learning and classification to *make it clear* whether the long-tailed recognition ability is achieved from better representation or more balanced classifier, I recommend that authors show us some visualization of the feature map besides number on performance. Because I am confused and difficult to image what *better representation* actually looks like. The authors conduct experiment with ResNeXt-{10,50,101,151}, and mainly use ResNeXt-50 for analysis. Will other networks get similar results as that of ResNeXt-50 shown in Figure 1? When showing the results, like Figure 1, 2 and Table 2, it would be better to mention the parameters chosen for 	au-normalization and other methods. Conclusion: I tend to accept this paper since it is interesting and renews our understanding of the long-tailed recognition ability of neural network and sampling strategies. What*s more, he experiment is comprehensive and rigorous.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Decoupling Representation and Classifier for Long-Tailed Recognition
The long-tail distribution of the visual world poses great challenges for deep learning based classification models on how to handle the class imbalance problem. Existing solutions usually involve class-balancing strategies, e.g., by loss re-weighting, data re-sampling, or transfer learning from head- to tail-classes, but all of them adhere to the scheme of jointly learning representations and classifiers. In this work, we decouple the learning procedure into representation learning and classification, and systematically explore how different balancing strategies affect them for long-tailed recognition. The findings are surprising: (1) data imbalance might not be an issue in learning high-quality representations; (2) with representations learned with the simplest instance-balanced (natural) sampling, it is also possible to achieve strong long-tailed recognition ability at little to no cost by adjusting only the classifier. We conduct extensive experiments and set new state-of-the-art performance on common long-tailed benchmarks like ImageNet-LT, Places-LT and iNaturalist, showing that it is possible to outperform carefully designed losses, sampling strategies, even complex modules with memory, by using a straightforward approach that decouples representation and classification.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



The paper tries to handle the class imbalance problem by decoupling the learning process into representation learning and classification, in contrast to the current methods that jointly learn both of them. They comprehensively study several sampling methods for representation learning and different strategies for classification. They find that instance-balanced sampling gives the best representation, and simply adjusting the classifier will equip the model with long-tailed recognition ability. They achieve start of art on long-tailed data (ImageNet-LT, Places-LT and iNaturalist). In general, this is paper is an interesting paper. The author propose that instance-balanced sampling already learns the best and most generalizable representations, which is out of common expectation. They perform extensive experiment to illustrate their points. --Writing: This paper is well written in English and is well structured. And there are two typos. One is in the second row of page 3, *… a more continuous decrease [in in] class labels …* and the other one is in the first paragraph of section 5.4, *… report state-of-art results [on on] three common long-tailed benchmarks …*. --Introduction and review: The authors do a comprehensive literature review, listing the main directions for solving the long-tailed recognition problem. They emphasis that these methods all jointly learn the representations and classifiers, which *make it unclear how the long-tailed recognition ability is achieved-is it from learning a better representation or by handling the data imbalance better via shifting classifier decision boundaries*. This motivate them to decouple representation learning and classification. --Experiment: Since this paper decouples the representation learning and classification to *make it clear* whether the long-tailed recognition ability is achieved from better representation or more balanced classifier, I recommend that authors show us some visualization of the feature map besides number on performance. Because I am confused and difficult to image what *better representation* actually looks like. The authors conduct experiment with ResNeXt-{10,50,101,151}, and mainly use ResNeXt-50 for analysis. Will other networks get similar results as that of ResNeXt-50 shown in Figure 1? When showing the results, like Figure 1, 2 and Table 2, it would be better to mention the parameters chosen for 	au-normalization and other methods. Conclusion: I tend to accept this paper since it is interesting and renews our understanding of the long-tailed recognition ability of neural network and sampling strategies. What*s more, he experiment is comprehensive and rigorous.
assistant
The paper tries to handle the class imbalance problem by decoupling the learning process into representation learning and classification, in contrast to the current methods that jointly learn both of them. They comprehensively study several sampling methods for representation learning and different strategies for classification. They find that instance-balanced sampling gives the best representation, and simply adjusting the classifier will equip the model with long-tailed recognition ability. They achieve start of art on long-tailed data (ImageNet-LT, Places-LT and iNaturalist). In general, this is paper is an interesting paper. The author propose that instance-balanced sampling already learns the best and most generalizable representations, which is out of common expectation. They perform extensive experiment to illustrate their points. --Writing: This paper is well written in English and is well structured. And there are two typos. One is in the second row of page 3, *… a more continuous decrease [in in] class labels …* and the other one is in the first paragraph of section 5.4, *… report state-of-art results [on on] three common long-tailed benchmarks …*. --Introduction and review: The authors do a comprehensive literature review, listing the main directions for solving the long-tailed recognition problem. They emphasis that these methods all jointly learn the representations and classifiers, which *make it unclear how the long-tailed recognition ability is achieved-is it from learning a better representation or by handling the data imbalance better via shifting classifier decision boundaries*. This motivate them to decouple representation learning and classification. --Experiment: Since this paper decouples the representation learning and classification to *make it clear* whether the long-tailed recognition ability is achieved from better representation or more balanced classifier, I recommend that authors show us some visualization of the feature map besides number on performance. Because I am confused and difficult to image what *better representation* actually looks like. The authors conduct experiment with ResNeXt-{10,50,101,151}, and mainly use ResNeXt-50 for analysis. Will other networks get similar results as that of ResNeXt-50 shown in Figure 1? When showing the results, like Figure 1, 2 and Table 2, it would be better to mention the parameters chosen for 	au-normalization and other methods. Conclusion: I tend to accept this paper since it is interesting and renews our understanding of the long-tailed recognition ability of neural network and sampling strategies. What*s more, he experiment is comprehensive and rigorous.