Modeling Real-Time Interactive Conversations as Timed Diarized Transcripts
<|reference_start|>Modeling Real-Time Interactive Conversations as Timed Diarized Transcripts: Chatbots built upon language models have exploded in popularity, but they have largely been limited to synchronous, turn-by-turn dialogues. In this paper we present a simple yet general method to simulate real-time interactive conversations using pretrained text-only language models, by modeling timed diarized transcripts and decoding them with causal rejection sampling. We demonstrate the promise of this method with two case studies: instant messenger dialogues and spoken conversations, which require generation at about 30 tok/s and 20 tok/s respectively to maintain real-time interactivity. These capabilities can be added into language models using relatively little data and run on commodity hardware.<|reference_end|>
@article{tanzer2024modeling,
    title={Modeling Real-Time Interactive Conversations as Timed Diarized
  Transcripts},
    author={Garrett Tanzer, Gustaf Ahdritz, Luke Melas-Kyriazi},
    journal={arXiv preprint arXiv:2405.13203},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.13203},
    primaryClass={cs.LG cs.CL}
}