ObfuscaTune: Obfuscated Offsite Fine-tuning and Inference of Proprietary LLMs on Private Datasets
<|reference_start|>ObfuscaTune: Obfuscated Offsite Fine-tuning and Inference of Proprietary LLMs on Private Datasets: This work addresses the timely yet underexplored problem of performing inference and finetuning of a proprietary LLM owned by a model provider entity on the confidential/private data of another data owner entity, in a way that ensures the confidentiality of both the model and the data. Hereby, the finetuning is conducted offsite, i.e., on the computation infrastructure of a third-party cloud provider. We tackle this problem by proposing ObfuscaTune, a novel, efficient and fully utility-preserving approach that combines a simple yet effective obfuscation technique with an efficient usage of confidential computing (only 5% of the model parameters are placed on TEE). We empirically demonstrate the effectiveness of ObfuscaTune by validating it on GPT-2 models with different sizes on four NLP benchmark datasets. Finally, we compare to a na\"ive version of our approach to highlight the necessity of using random matrices with low condition numbers in our approach to reduce errors induced by the obfuscation.<|reference_end|>
@article{frikha2024obfuscatune:,
    title={ObfuscaTune: Obfuscated Offsite Fine-tuning and Inference of Proprietary
  LLMs on Private Datasets},
    author={Ahmed Frikha, Nassim Walha, Ricardo Mendes, Krishna Kanth Nakka, Xue
  Jiang, Xuebing Zhou},
    journal={arXiv preprint arXiv:2407.02960},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.02960},
    primaryClass={cs.CR cs.AI cs.CL cs.LG}
}