Fine-Tuning and Deploying Large Language Models Over Edges: Issues and Approaches
<|reference_start|>Fine-Tuning and Deploying Large Language Models Over Edges: Issues and Approaches: Since the invention of GPT2--1.5B in 2019, large language models (LLMs) have transitioned from specialized models to versatile foundation models. The LLMs exhibit impressive zero-shot ability, however, require fine-tuning on local datasets and significant resources for deployment. Traditional fine-tuning techniques with the first-order optimizers require substantial GPU memory that exceeds mainstream hardware capability. Therefore, memory-efficient methods are motivated to be investigated. Model compression techniques can reduce energy consumption, operational costs, and environmental impact so that to support sustainable artificial intelligence advancements. Additionally, large-scale foundation models have expanded to create images, audio, videos, and multi-modal contents, further emphasizing the need for efficient deployment. Therefore, we are motivated to present a comprehensive overview of the prevalent memory-efficient fine-tuning methods over the network edge. We also review the state-of-the-art literatures on model compression to provide a vision on deploying LLMs over the network edge.<|reference_end|>
@article{dong2024fine-tuning,
    title={Fine-Tuning and Deploying Large Language Models Over Edges: Issues and
  Approaches},
    author={Yanjie Dong, Haijun Zhang, Chengming Li, Song Guo, Victor C. M. Leung,
  Xiping Hu},
    journal={arXiv preprint arXiv:2408.10691},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.10691},
    primaryClass={cs.AI}
}