FIFO-Diffusion: Generating Infinite Videos from Text without Training
<|reference_start|>FIFO-Diffusion: Generating Infinite Videos from Text without Training: We propose a novel inference technique based on a pretrained diffusion model for text-conditional video generation. Our approach, called FIFO-Diffusion, is conceptually capable of generating infinitely long videos without additional training. This is achieved by iteratively performing diagonal denoising, which concurrently processes a series of consecutive frames with increasing noise levels in a queue; our method dequeues a fully denoised frame at the head while enqueuing a new random noise frame at the tail. However, diagonal denoising is a double-edged sword as the frames near the tail can take advantage of cleaner ones by forward reference but such a strategy induces the discrepancy between training and inference. Hence, we introduce latent partitioning to reduce the training-inference gap and lookahead denoising to leverage the benefit of forward referencing. Practically, FIFO-Diffusion consumes a constant amount of memory regardless of the target video length given a baseline model, while well-suited for parallel inference on multiple GPUs. We have demonstrated the promising results and effectiveness of the proposed methods on existing text-to-video generation baselines. Generated video samples and source codes are available at our project page.<|reference_end|>
@article{kim2024fifo-diffusion:,
    title={FIFO-Diffusion: Generating Infinite Videos from Text without Training},
    author={Jihwan Kim, Junoh Kang, Jinyoung Choi, Bohyung Han},
    journal={NeurIPS 2024},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.11473},
    primaryClass={cs.CV cs.AI}
}