MPC-Minimized Secure LLM Inference
<|reference_start|>MPC-Minimized Secure LLM Inference: Many inference services based on large language models (LLMs) pose a privacy concern, either revealing user prompts to the service or the proprietary weights to the user. Secure inference offers a solution to this problem through secure multi-party computation (MPC), however, it is still impractical for modern LLM workload due to the large overhead imposed by MPC. To address this overhead, we propose Marill, a framework that adapts LLM fine-tuning to minimize MPC usage during secure inference. Marill introduces high-level architectural changes during fine-tuning that significantly reduce the number of expensive operations needed within MPC during inference, by removing some and relocating others outside MPC without compromising security. As a result, Marill-generated models are more efficient across all secure inference protocols and our approach complements MPC-friendly approximations for such operations. Compared to standard fine-tuning, Marill results in 3.6-11.3x better runtime and 2.4-6.9x better communication during secure inference across various MPC settings, while typically preserving over 90% performance across downstream tasks.<|reference_end|>
@article{rathee2024mpc-minimized,
    title={MPC-Minimized Secure LLM Inference},
    author={Deevashwer Rathee, Dacheng Li, Ion Stoica, Hao Zhang, Raluca Popa},
    journal={arXiv preprint arXiv:2408.03561},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.03561},
    primaryClass={cs.CR cs.AI cs.LG}
}