PocketLLM: Enabling On-Device Fine-Tuning for Personalized LLMs
<|reference_start|>PocketLLM: Enabling On-Device Fine-Tuning for Personalized LLMs: Recent advancements in large language models (LLMs) have indeed showcased their impressive capabilities. On mobile devices, the wealth of valuable, non-public data generated daily holds great promise for locally fine-tuning personalized LLMs, while maintaining privacy through on-device processing. However, the constraints of mobile device resources pose challenges to direct on-device LLM fine-tuning, mainly due to the memory-intensive nature of derivative-based optimization required for saving gradients and optimizer states. To tackle this, we propose employing derivative-free optimization techniques to enable on-device fine-tuning of LLM, even on memory-limited mobile devices. Empirical results demonstrate that the RoBERTa-large model and OPT-1.3B can be fine-tuned locally on the OPPO Reno 6 smartphone using around 4GB and 6.5GB of memory respectively, using derivative-free optimization techniques. This highlights the feasibility of on-device LLM fine-tuning on mobile devices, paving the way for personalized LLMs on resource-constrained devices while safeguarding data privacy.<|reference_end|>
@article{peng2024pocketllm:,
    title={PocketLLM: Enabling On-Device Fine-Tuning for Personalized LLMs},
    author={Dan Peng, Zhihui Fu, Jun Wang},
    journal={arXiv preprint arXiv:2407.01031},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.01031},
    primaryClass={cs.LG cs.CL}
}