title: LumosFlow: Motion-Guided Long Video Generation
summary: Long video generation has gained increasing attention due to its widespread
  applications in fields such as entertainment and simulation. Despite advances,
  synthesizing temporally coherent and visually compelling long sequences remains
  a formidable challenge. Conventional approaches often synthesize long videos by
  sequentially generating and concatenating short clips, or generating key frames
  and then interpolate the intermediate frames in a hierarchical manner. However,
  both of them still remain significant challenges, leading to issues such as
  temporal repetition or unnatural transitions. In this paper, we revisit the
  hierarchical long video generation pipeline and introduce LumosFlow, a
  framework introduce motion guidance explicitly. Specifically, we first employ
  the Large Motion Text-to-Video Diffusion Model (LMTV-DM) to generate key frames
  with larger motion intervals, thereby ensuring content diversity in the
  generated long videos. Given the complexity of interpolating contextual
  transitions between key frames, we further decompose the intermediate frame
  interpolation into motion generation and post-hoc refinement. For each pair of
  key frames, the Latent Optical Flow Diffusion Model (LOF-DM) synthesizes
  complex and large-motion optical flows, while MotionControlNet subsequently
  refines the warped results to enhance quality and guide intermediate frame
  generation. Compared with traditional video frame interpolation, we achieve 15x
  interpolation, ensuring reasonable and continuous motion between adjacent
  frames. Experiments show that our method can generate long videos with
  consistent motion and appearance. Code and models will be made publicly
  available upon acceptance. Our project page:
  https://jiahaochen1.github.io/LumosFlow/
paper_authors._id: 683fbc32917306517315589f
paper_authors.name: Jiahao Chen
paper_authors._id: 683fbc3291730651731558a0
paper_authors.name: Hangjie Yuan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-05T07:28:23.685Z
paper_authors._id: 649d54b314afbb10ce2a9eeb
paper_authors.avatarUrl: /avatars/15c325d8c2273ff63569f23015e98486.svg
paper_authors.fullname: Hangjie Yuan
paper_authors.type: user
paper_authors.user: JacobYuan
paper_authors._id: 683fbc3291730651731558a1
paper_authors.name: Yichen Qian
paper_authors._id: 683fbc3291730651731558a2
paper_authors.name: Jingyun Liang
paper_authors._id: 683fbc3291730651731558a3
paper_authors.name: Jiazheng Xing
paper_authors._id: 683fbc3291730651731558a4
paper_authors.name: Pengwei Liu
paper_authors._id: 683fbc3291730651731558a5
paper_authors.name: Weihua Chen
paper_authors._id: 683fbc3291730651731558a6
paper_authors.name: Fan Wang
paper_authors._id: 683fbc3291730651731558a7
paper_authors.name: Bing Su
paper_title: LumosFlow: Motion-Guided Long Video Generation
paper_summary: Long video generation has gained increasing attention due to its widespread
  applications in fields such as entertainment and simulation. Despite advances,
  synthesizing temporally coherent and visually compelling long sequences remains
  a formidable challenge. Conventional approaches often synthesize long videos by
  sequentially generating and concatenating short clips, or generating key frames
  and then interpolate the intermediate frames in a hierarchical manner. However,
  both of them still remain significant challenges, leading to issues such as
  temporal repetition or unnatural transitions. In this paper, we revisit the
  hierarchical long video generation pipeline and introduce LumosFlow, a
  framework introduce motion guidance explicitly. Specifically, we first employ
  the Large Motion Text-to-Video Diffusion Model (LMTV-DM) to generate key frames
  with larger motion intervals, thereby ensuring content diversity in the
  generated long videos. Given the complexity of interpolating contextual
  transitions between key frames, we further decompose the intermediate frame
  interpolation into motion generation and post-hoc refinement. For each pair of
  key frames, the Latent Optical Flow Diffusion Model (LOF-DM) synthesizes
  complex and large-motion optical flows, while MotionControlNet subsequently
  refines the warped results to enhance quality and guide intermediate frame
  generation. Compared with traditional video frame interpolation, we achieve 15x
  interpolation, ensuring reasonable and continuous motion between adjacent
  frames. Experiments show that our method can generate long videos with
  consistent motion and appearance. Code and models will be made publicly
  available upon acceptance. Our project page:
  https://jiahaochen1.github.io/LumosFlow/
paper_ai_summary: LumosFlow uses LMTV-DM for key frame generation and LOF-DM followed by MotionControlNet for smooth intermediate frame interpolation, ensuring temporally coherent long video generation.
paper_ai_keywords: Large Motion Text-to-Video Diffusion Model LMTV-DM Latent Optical Flow Diffusion Model LOF-DM MotionControlNet optical flows frame interpolation key frames long video generation motion guidance synthetic long videos