Enabling Resource-Efficient On-Device Fine-Tuning of LLMs Using Only Inference Engines
<|reference_start|>Enabling Resource-Efficient On-Device Fine-Tuning of LLMs Using Only Inference Engines: Large Language Models (LLMs) have demonstrated exceptional performance in automating various tasks, such as text generation and summarization. Currently LLMs are trained and fine-tuned on large cloud server. Deploying and fine-tuning these models on resource-constrained edge devices remains a significant challenge due to their substantial memory and computational requirements. This paper introduces a resource-efficient zeroth-order optimization approach that lowers the barriers for fine-tuning LLMs in such constrained environments. Our method features a parallelized randomized gradient estimation (P-RGE) technique, which performs gradient estimation with high parallel efficiency. P-RGE leverages outer-loop and inner-loop parallelization to perform multiple function queries and forward passes in parallel, reducing the wall-clock end-to-end training time. By integrating this technique with parameter-efficient fine-tuning methods (e.g., LoRA) and on-device inference engines (e.g., ExecuTorch), we demonstrate efficient fine-tuning of LLMs on both server-side and edge devices. Experiments show that P-RGE achieves significant runtime speedups and memory savings while maintaining fine-tuning accuracy, which paves the way for more practical deployment of LLMs in real-time, on-device applications.<|reference_end|>
@article{gao2024enabling,
    title={Enabling Efficient On-Device Fine-Tuning of LLMs Using Only Inference
  Engines},
    author={Lei Gao, Amir Ziashahabi, Yue Niu, Salman Avestimehr, Murali Annavaram},
    journal={arXiv preprint arXiv:2409.15520},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.15520},
    primaryClass={cs.LG cs.DC}
}