RobustSentEmbed: Robust Sentence Embeddings Using Adversarial Self-Supervised Contrastive Learning
<|reference_start|>RobustSentEmbed: Robust Sentence Embeddings Using Adversarial Self-Supervised Contrastive Learning: Pre-trained language models (PLMs) have consistently demonstrated outstanding performance across a diverse spectrum of natural language processing tasks. Nevertheless, despite their success with unseen data, current PLM-based representations often exhibit poor robustness in adversarial settings. In this paper, we introduce RobustSentEmbed, a self-supervised sentence embedding framework designed to improve both generalization and robustness in diverse text representation tasks and against a diverse set of adversarial attacks. Through the generation of high-risk adversarial perturbations and their utilization in a novel objective function, RobustSentEmbed adeptly learns high-quality and robust sentence embeddings. Our experiments confirm the superiority of RobustSentEmbed over state-of-the-art representations. Specifically, Our framework achieves a significant reduction in the success rate of various adversarial attacks, notably reducing the BERTAttack success rate by almost half (from 75.51\% to 38.81\%). The framework also yields improvements of 1.59\% and 0.23\% in semantic textual similarity tasks and various transfer tasks, respectively.<|reference_end|>
@article{asl2024robustsentembed:,
    title={RobustSentEmbed: Robust Sentence Embeddings Using Adversarial
  Self-Supervised Contrastive Learning},
    author={Javad Rafiei Asl, Prajwal Panzade, Eduardo Blanco, Daniel Takabi,
  Zhipeng Cai},
    journal={arXiv preprint arXiv:2403.11082},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.11082},
    primaryClass={cs.CL cs.AI cs.LG}
}