Enabling Real-Time Conversations with Minimal Training Costs
<|reference_start|>Enabling Real-Time Conversations with Minimal Training Costs: Large language models (LLMs) have demonstrated the ability to improve human efficiency through conversational interactions. Conventional LLM-powered dialogue systems, operating on a turn-based paradigm, preclude real-time interaction during response generation. To address this limitation, researchers have proposed duplex models. These models can dynamically adapt to user input, facilitating real-time interactive feedback. However, these methods typically require substantial computational resources to acquire the ability. To reduce overhead, this paper presents a new duplex decoding approach that enhances LLMs with duplex ability, requiring minimal additional training. Specifically, our method employs parallel decoding of queries and responses in conversations, effectively implementing a channel-division-multiplexing decoding strategy. Experimental results indicate that our proposed method significantly enhances the naturalness and human-likeness of user-AI interactions with minimal training costs.<|reference_end|>
@article{xu2024enabling,
    title={Enabling Real-Time Conversations with Minimal Training Costs},
    author={Wang Xu, Shuo Wang, Weilin Zhao, Xu Han, Yukun Yan, Yudi Zhang, Zhe
  Tao, Zhiyuan Liu, Wanxiang Che},
    journal={arXiv preprint arXiv:2409.11727},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.11727},
    primaryClass={cs.CL}
}