Personalized LoRA for Human-Centered Text Understanding
<|reference_start|>Personalized LoRA for Human-Centered Text Understanding: Effectively and efficiently adapting a pre-trained language model (PLM) for human-centered text understanding (HCTU) is challenging since user tokens are million-level in most personalized applications and do not have concrete explicit semantics. A standard and parameter-efficient approach (e.g., LoRA) necessitates memorizing numerous suits of adapters for each user. In this work, we introduce a personalized LoRA (PLoRA) with a plug-and-play (PnP) framework for the HCTU task. PLoRA is effective, parameter-efficient, and dynamically deploying in PLMs. Moreover, a personalized dropout and a mutual information maximizing strategies are adopted and hence the proposed PLoRA can be well adapted to few/zero-shot learning scenarios for the cold-start issue. Experiments conducted on four benchmark datasets show that the proposed method outperforms existing methods in full/few/zero-shot learning scenarios for the HCTU task, even though it has fewer trainable parameters. For reproducibility, the code for this paper is available at: https://github.com/yoyo-yun/PLoRA.<|reference_end|>
@article{zhang2024personalized,
    title={Personalized LoRA for Human-Centered Text Understanding},
    author={You Zhang, Jin Wang, Liang-Chih Yu, Dan Xu, Xuejie Zhang},
    journal={arXiv preprint arXiv:2403.06208},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.06208},
    primaryClass={cs.CL}
}