When resampling/reweighting improves feature learning in imbalanced classification?: A toy-model study
<|reference_start|>When resampling/reweighting improves feature learning in imbalanced classification?: A toy-model study: A toy model of binary classification is studied with the aim of clarifying the class-wise resampling/reweighting effect on the feature learning performance under the presence of class imbalance. In the analysis, a high-dimensional limit of the feature is taken while keeping the dataset size ratio against the feature dimension finite and the non-rigorous replica method from statistical mechanics is employed. The result shows that there exists a case in which the no resampling/reweighting situation gives the best feature learning performance irrespectively of the choice of losses or classifiers, supporting recent findings in Cao et al. (2019); Kang et al. (2019). It is also revealed that the key of the result is the symmetry of the loss and the problem setting. Inspired by this, we propose a further simplified model exhibiting the same property for the multiclass setting. These clarify when the class-wise resampling/reweighting becomes effective in imbalanced classification.<|reference_end|>
@article{obuchi2024when,
    title={When resampling/reweighting improves feature learning in imbalanced
  classification?: A toy-model study},
    author={Tomoyuki Obuchi, Toshiyuki Tanaka},
    journal={arXiv preprint arXiv:2409.05598},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.05598},
    primaryClass={stat.ML cond-mat.dis-nn cs.IT cs.LG math.IT}
}