LLM Inference Serving: Survey of Recent Advances and Opportunities
<|reference_start|>LLM Inference Serving: Survey of Recent Advances and Opportunities: This survey offers a comprehensive overview of recent advancements in Large Language Model (LLM) serving systems, focusing on research since the year 2023. We specifically examine system-level enhancements that improve performance and efficiency without altering the core LLM decoding mechanisms. By selecting and reviewing high-quality papers from prestigious ML and system venues, we highlight key innovations and practical considerations for deploying and scaling LLMs in real-world production environments. This survey serves as a valuable resource for LLM practitioners seeking to stay abreast of the latest developments in this rapidly evolving field.<|reference_end|>
@article{li2024llm,
    title={LLM Inference Serving: Survey of Recent Advances and Opportunities},
    author={Baolin Li, Yankai Jiang, Vijay Gadepally, Devesh Tiwari},
    journal={arXiv preprint arXiv:2407.12391},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.12391},
    primaryClass={cs.DC cs.AI}
}