Swift Sampler: Efficient Learning of Sampler by 10 Parameters
<|reference_start|>Swift Sampler: Efficient Learning of Sampler by 10 Parameters: Data selection is essential for training deep learning models. An effective data sampler assigns proper sampling probability for training data and helps the model converge to a good local minimum with high performance. Previous studies in data sampling are mainly based on heuristic rules or learning through a huge amount of time-consuming trials. In this paper, we propose an automatic \textbf{swift sampler} search algorithm, \textbf{SS}, to explore automatically learning effective samplers efficiently. In particular, \textbf{SS} utilizes a novel formulation to map a sampler to a low dimension of hyper-parameters and uses an approximated local minimum to quickly examine the quality of a sampler. Benefiting from its low computational expense, \textbf{SS} can be applied on large-scale data sets with high efficiency. Comprehensive experiments on various tasks demonstrate that \textbf{SS} powered sampling can achieve obvious improvements (e.g., 1.5\% on ImageNet) and transfer among different neural networks. Project page: https://github.com/Alexander-Yao/Swift-Sampler.<|reference_end|>
@article{yao2024swift,
    title={Swift Sampler: Efficient Learning of Sampler by 10 Parameters},
    author={Jiawei Yao, Chuming Li, Canran Xiao},
    journal={arXiv preprint arXiv:2410.05578},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.05578},
    primaryClass={cs.LG cs.AI}
}