Batch-in-Batch: a new adversarial training framework for initial perturbation and sample selection
<|reference_start|>Batch-in-Batch: a new adversarial training framework for initial perturbation and sample selection: Adversarial training methods commonly generate independent initial perturbation for adversarial samples from a simple uniform distribution, and obtain the training batch for the classifier without selection. In this work, we propose a simple yet effective training framework called Batch-in-Batch (BB) to enhance models robustness. It involves specifically a joint construction of initial values that could simultaneously generates $m$ sets of perturbations from the original batch set to provide more diversity for adversarial samples; and also includes various sample selection strategies that enable the trained models to have smoother losses and avoid overconfident outputs. Through extensive experiments on three benchmark datasets (CIFAR-10, SVHN, CIFAR-100) with two networks (PreActResNet18 and WideResNet28-10) that are used in both the single-step (Noise-Fast Gradient Sign Method, N-FGSM) and multi-step (Projected Gradient Descent, PGD-10) adversarial training, we show that models trained within the BB framework consistently have higher adversarial accuracy across various adversarial settings, notably achieving over a 13% improvement on the SVHN dataset with an attack radius of 8/255 compared to the N-FGSM baseline model. Furthermore, experimental analysis of the efficiency of both the proposed initial perturbation method and sample selection strategies validates our insights. Finally, we show that our framework is cost-effective in terms of computational resources, even with a relatively large value of $m$.<|reference_end|>
@article{wu2024batch-in-batch:,
    title={Batch-in-Batch: a new adversarial training framework for initial
  perturbation and sample selection},
    author={Yinting Wu (1), Pai Peng (2), Bo Cai (3), Le Li (1). ((1) School of
  Mathematics and Statistics, and Key Lab NAA--MOE, Central China Normal
  University, (2) School of Mathematics and Computer Science, Jianghan
  University, (3) Key Laboratory of Aerospace Information Security and Trusted
  Computing, Ministry of Education, and School of Cyber Science and
  Engineering, Wuhan University)},
    journal={arXiv preprint arXiv:2406.04070},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.04070},
    primaryClass={cs.LG cs.AI}
}