Decoupling Representation and Classifier for Long-Tailed Recognition
The long-tail distribution of the visual world poses great challenges for deep learning based classification models on how to handle the class imbalance problem. Existing solutions usually involve class-balancing strategies, e.g., by loss re-weighting, data re-sampling, or transfer learning from head- to tail-classes, but all of them adhere to the scheme of jointly learning representations and classifiers. In this work, we decouple the learning procedure into representation learning and classification, and systematically explore how different balancing strategies affect them for long-tailed recognition. The findings are surprising: (1) data imbalance might not be an issue in learning high-quality representations; (2) with representations learned with the simplest instance-balanced (natural) sampling, it is also possible to achieve strong long-tailed recognition ability at little to no cost by adjusting only the classifier. We conduct extensive experiments and set new state-of-the-art performance on common long-tailed benchmarks like ImageNet-LT, Places-LT and iNaturalist, showing that it is possible to outperform carefully designed losses, sampling strategies, even complex modules with memory, by using a straightforward approach that decouples representation and classification.
This paper proposes to tackle long-tailed classification by treating separately the representation learning and the creation of a classifier for test time. They evaluate their method on several standard long-tailed classification datasets like ImageNet-LT or Places-LT. Pros: * Very well presented and clear * Thorough experiments with baselines and comparisons with competitors * Novel and efficient approach of redesigning the classifier as a post-processing step after the representation training Cons: * I did not find any single value of the *temperature* coefficient that you use for the different datasets! According to Fig 2, it should be around 0.7 for ImageNet-LT but you should clearly specify the used values for all the datasets. For reproducibility. It is also important to know it as it has an impact on how useful is this approach in practice. Because if the coefficients are very different for all the datasets, then the method requires a validation set to find this hyperparameter. * As middle point between NCM and cRT, you could also train a cosine classifier as done in the paper that you cite *Dynamic few-shot visual learning without forgetting* by Gidaris et al. There is pytorch code for it online. I am leaning towards acceptance as the method is clear, easy to implement, well studied through the experiments and has good results on standard benchmarks. The paper also provides interesting insights about long-tailed recognition in general like the effect of the different samplings with the proposed method.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Decoupling Representation and Classifier for Long-Tailed Recognition
The long-tail distribution of the visual world poses great challenges for deep learning based classification models on how to handle the class imbalance problem. Existing solutions usually involve class-balancing strategies, e.g., by loss re-weighting, data re-sampling, or transfer learning from head- to tail-classes, but all of them adhere to the scheme of jointly learning representations and classifiers. In this work, we decouple the learning procedure into representation learning and classification, and systematically explore how different balancing strategies affect them for long-tailed recognition. The findings are surprising: (1) data imbalance might not be an issue in learning high-quality representations; (2) with representations learned with the simplest instance-balanced (natural) sampling, it is also possible to achieve strong long-tailed recognition ability at little to no cost by adjusting only the classifier. We conduct extensive experiments and set new state-of-the-art performance on common long-tailed benchmarks like ImageNet-LT, Places-LT and iNaturalist, showing that it is possible to outperform carefully designed losses, sampling strategies, even complex modules with memory, by using a straightforward approach that decouples representation and classification.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



This paper proposes to tackle long-tailed classification by treating separately the representation learning and the creation of a classifier for test time. They evaluate their method on several standard long-tailed classification datasets like ImageNet-LT or Places-LT. Pros: * Very well presented and clear * Thorough experiments with baselines and comparisons with competitors * Novel and efficient approach of redesigning the classifier as a post-processing step after the representation training Cons: * I did not find any single value of the *temperature* coefficient that you use for the different datasets! According to Fig 2, it should be around 0.7 for ImageNet-LT but you should clearly specify the used values for all the datasets. For reproducibility. It is also important to know it as it has an impact on how useful is this approach in practice. Because if the coefficients are very different for all the datasets, then the method requires a validation set to find this hyperparameter. * As middle point between NCM and cRT, you could also train a cosine classifier as done in the paper that you cite *Dynamic few-shot visual learning without forgetting* by Gidaris et al. There is pytorch code for it online. I am leaning towards acceptance as the method is clear, easy to implement, well studied through the experiments and has good results on standard benchmarks. The paper also provides interesting insights about long-tailed recognition in general like the effect of the different samplings with the proposed method.
assistant
This paper proposes to tackle long-tailed classification by treating separately the representation learning and the creation of a classifier for test time. They evaluate their method on several standard long-tailed classification datasets like ImageNet-LT or Places-LT. Pros: * Very well presented and clear * Thorough experiments with baselines and comparisons with competitors * Novel and efficient approach of redesigning the classifier as a post-processing step after the representation training Cons: * I did not find any single value of the *temperature* coefficient that you use for the different datasets! According to Fig 2, it should be around 0.7 for ImageNet-LT but you should clearly specify the used values for all the datasets. For reproducibility. It is also important to know it as it has an impact on how useful is this approach in practice. Because if the coefficients are very different for all the datasets, then the method requires a validation set to find this hyperparameter. * As middle point between NCM and cRT, you could also train a cosine classifier as done in the paper that you cite *Dynamic few-shot visual learning without forgetting* by Gidaris et al. There is pytorch code for it online. I am leaning towards acceptance as the method is clear, easy to implement, well studied through the experiments and has good results on standard benchmarks. The paper also provides interesting insights about long-tailed recognition in general like the effect of the different samplings with the proposed method.