Amplifying Training Data Exposure through Fine-Tuning with Pseudo-Labeled Memberships
<|reference_start|>Amplifying Training Data Exposure through Fine-Tuning with Pseudo-Labeled Memberships: Neural language models (LMs) are vulnerable to training data extraction attacks due to data memorization. This paper introduces a novel attack scenario wherein an attacker adversarially fine-tunes pre-trained LMs to amplify the exposure of the original training data. This strategy differs from prior studies by aiming to intensify the LM's retention of its pre-training dataset. To achieve this, the attacker needs to collect generated texts that are closely aligned with the pre-training data. However, without knowledge of the actual dataset, quantifying the amount of pre-training data within generated texts is challenging. To address this, we propose the use of pseudo-labels for these generated texts, leveraging membership approximations indicated by machine-generated probabilities from the target LM. We subsequently fine-tune the LM to favor generations with higher likelihoods of originating from the pre-training data, based on their membership probabilities. Our empirical findings indicate a remarkable outcome: LMs with over 1B parameters exhibit a four to eight-fold increase in training data exposure. We discuss potential mitigations and suggest future research directions.<|reference_end|>
@article{oh2024amplifying,
    title={Amplifying Training Data Exposure through Fine-Tuning with
  Pseudo-Labeled Memberships},
    author={Myung Gyo Oh, Hong Eun Ahn, Leo Hyun Park, Taekyoung Kwon},
    journal={arXiv preprint arXiv:2402.12189},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.12189},
    primaryClass={cs.CL cs.CR cs.LG}
}