title: ^RFLAV: Rolling Flow matching for infinite Audio Video generation
summary: Joint audio-video (AV) generation is still a significant challenge in
  generative AI, primarily due to three critical requirements: quality of the
  generated samples, seamless multimodal synchronization and temporal coherence,
  with audio tracks that match the visual data and vice versa, and limitless
  video duration. In this paper, we present , a novel transformer-based
  architecture that addresses all the key challenges of AV generation. We explore
  three distinct cross modality interaction modules, with our lightweight
  temporal fusion module emerging as the most effective and computationally
  efficient approach for aligning audio and visual modalities. Our experimental
  results demonstrate that  outperforms existing state-of-the-art models
  in multimodal AV generation tasks. Our code and checkpoints are available at
  https://github.com/ErgastiAlex/R-FLAV.
paper_authors._id: 67d140378cb4592900a1a75e
paper_authors.name: Alex Ergasti
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-12T08:36:03.978Z
paper_authors._id: 66895b3d41fcf83c026b5dca
paper_authors.avatarUrl: /avatars/f1104041ee3445024f05d5d0b4d1550b.svg
paper_authors.fullname: Alex Ergasti
paper_authors.type: user
paper_authors.user: MaverickAlex
paper_authors._id: 67d140378cb4592900a1a75f
paper_authors.name: Giuseppe Gabriele Tarollo
paper_authors._id: 67d140378cb4592900a1a760
paper_authors.name: Filippo Botti
paper_authors._id: 67d140378cb4592900a1a761
paper_authors.name: Tomaso Fontanini
paper_authors._id: 67d140378cb4592900a1a762
paper_authors.name: Claudio Ferrari
paper_authors._id: 67d140378cb4592900a1a763
paper_authors.name: Massimo Bertozzi
paper_authors._id: 67d140378cb4592900a1a764
paper_authors.name: Andrea Prati
paper_title: ^RFLAV: Rolling Flow matching for infinite Audio Video generation
paper_summary: Joint audio-video (AV) generation is still a significant challenge in
  generative AI, primarily due to three critical requirements: quality of the
  generated samples, seamless multimodal synchronization and temporal coherence,
  with audio tracks that match the visual data and vice versa, and limitless
  video duration. In this paper, we present , a novel transformer-based
  architecture that addresses all the key challenges of AV generation. We explore
  three distinct cross modality interaction modules, with our lightweight
  temporal fusion module emerging as the most effective and computationally
  efficient approach for aligning audio and visual modalities. Our experimental
  results demonstrate that  outperforms existing state-of-the-art models
  in multimodal AV generation tasks. Our code and checkpoints are available at
  https://github.com/ErgastiAlex/R-FLAV.
paper_ai_keywords: transformer-based architecture cross modality interaction modules lightweight temporal fusion module audio and visual modalities multimodal AV generation tasks