MHLR: Moving Haar Learning Rate Scheduler for Large-scale Face Recognition Training with One GPU
<|reference_start|>MHLR: Moving Haar Learning Rate Scheduler for Large-scale Face Recognition Training with One GPU: Face recognition (FR) has seen significant advancements due to the utilization of large-scale datasets. Training deep FR models on large-scale datasets with multiple GPUs is now a common practice. In fact, computing power has evolved into a foundational and indispensable resource in the area of deep learning. It is nearly impossible to train a deep FR model without holding adequate hardware resources. Recognizing this challenge, some FR approaches have started exploring ways to reduce the time complexity of the fully-connected layer in FR models. Unlike other approaches, this paper introduces a simple yet highly effective approach, Moving Haar Learning Rate (MHLR) scheduler, for scheduling the learning rate promptly and accurately in the training process. MHLR supports large-scale FR training with only one GPU, which is able to accelerate the model to 1/4 of its original training time without sacrificing more than 1% accuracy. More specifically, MHLR only needs $30$ hours to train the model ResNet100 on the dataset WebFace12M containing more than 12M face images with 0.6M identities. Extensive experiments validate the efficiency and effectiveness of MHLR.<|reference_end|>
@article{gong2024mhlr:,
    title={MHLR: Moving Haar Learning Rate Scheduler for Large-scale Face
  Recognition Training with One GPU},
    author={Xueyuan Gong, Yain-whar Si, Zheng Zhang, Xiaochen Yuan, Ke Wang,
  Xinyuan Zhang, Cong Lin, and Xiaoxiang Liu},
    journal={arXiv preprint arXiv:2404.11118},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.11118},
    primaryClass={cs.CV}
}