title: Be-Your-Outpainter: Mastering Video Outpainting through Input-Specific
    Adaptation
summary: Video outpainting is a challenging task, aiming at generating video content
  outside the viewport of the input video while maintaining inter-frame and
  intra-frame consistency. Existing methods fall short in either generation
  quality or flexibility. We introduce MOTIA Mastering Video Outpainting Through
  Input-Specific Adaptation, a diffusion-based pipeline that leverages both the
  intrinsic data-specific patterns of the source video and the image/video
  generative prior for effective outpainting. MOTIA comprises two main phases:
  input-specific adaptation and pattern-aware outpainting. The input-specific
  adaptation phase involves conducting efficient and effective pseudo outpainting
  learning on the single-shot source video. This process encourages the model to
  identify and learn patterns within the source video, as well as bridging the
  gap between standard generative processes and outpainting. The subsequent
  phase, pattern-aware outpainting, is dedicated to the generalization of these
  learned patterns to generate outpainting outcomes. Additional strategies
  including spatial-aware insertion and noise travel are proposed to better
  leverage the diffusion model's generative prior and the acquired video patterns
  from source videos. Extensive evaluations underscore MOTIA's superiority,
  outperforming existing state-of-the-art methods in widely recognized
  benchmarks. Notably, these advancements are achieved without necessitating
  extensive, task-specific tuning.
paper_authors._id: 65fbba2ba9d83c79a62d0c9c
paper_authors.name: Fu-Yun Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-03-21T08:26:23.869Z
paper_authors._id: 63e9e92f20c109718713f5eb
paper_authors.avatarUrl: /avatars/9ff312e854d803e1a2e9e685a21d12f8.svg
paper_authors.fullname: Fu-Yun Wang
paper_authors.type: user
paper_authors.user: wangfuyun
paper_authors._id: 65fbba2ba9d83c79a62d0c9d
paper_authors.name: Xiaoshi Wu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-21T10:31:30.970Z
paper_authors._id: 6423f44d30b0e4ab36dd939d
paper_authors.avatarUrl: /avatars/9bbf6f00188d21d7bd5b678b955b9fc0.svg
paper_authors.fullname: Xiaoshi Wu
paper_authors.type: user
paper_authors.user: xswu
paper_authors._id: 65fbba2ba9d83c79a62d0c9e
paper_authors.name: Zhaoyang Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-21T10:31:51.493Z
paper_authors._id: 64c26e0e400274d129a10d5d
paper_authors.avatarUrl: /avatars/c1debf3d7f61c2ea01d9a25d7e332712.svg
paper_authors.fullname: Zhaoyang Huang
paper_authors.type: user
paper_authors.user: DrinkingCoder
paper_authors._id: 65fbba2ba9d83c79a62d0c9f
paper_authors.name: Xiaoyu Shi
paper_authors._id: 65fbba2ba9d83c79a62d0ca0
paper_authors.name: Dazhong Shen
paper_authors._id: 65fbba2ba9d83c79a62d0ca1
paper_authors.name: Guanglu Song
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-21T10:32:45.504Z
paper_authors._id: 636a2efffbc31ee68fee5c1c
paper_authors.avatarUrl: /avatars/21e119f23e1e6fe926567409e74d1a89.svg
paper_authors.fullname: Song
paper_authors.type: user
paper_authors.user: Guanglu
paper_authors._id: 65fbba2ba9d83c79a62d0ca2
paper_authors.name: Yu Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-21T10:33:26.329Z
paper_authors._id: 636de85cc4a7a729c164d2b5
paper_authors.avatarUrl: /avatars/3e281e547e1697e1c06805e7e63f3918.svg
paper_authors.fullname: Yu Liu
paper_authors.type: user
paper_authors.user: YuLiu
paper_authors._id: 65fbba2ba9d83c79a62d0ca3
paper_authors.name: Hongsheng Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-21T10:33:34.953Z
paper_authors._id: 65c04e9c27a5fdca81abcbd9
paper_authors.avatarUrl: /avatars/12a155683c824fa23da4a9e2bed4f64e.svg
paper_authors.fullname: Hongsheng LI
paper_authors.type: user
paper_authors.user: hsli-cuhk
paper_title: Be-Your-Outpainter: Mastering Video Outpainting through Input-Specific
    Adaptation
paper_summary: Video outpainting is a challenging task, aiming at generating video content
  outside the viewport of the input video while maintaining inter-frame and
  intra-frame consistency. Existing methods fall short in either generation
  quality or flexibility. We introduce MOTIA Mastering Video Outpainting Through
  Input-Specific Adaptation, a diffusion-based pipeline that leverages both the
  intrinsic data-specific patterns of the source video and the image/video
  generative prior for effective outpainting. MOTIA comprises two main phases:
  input-specific adaptation and pattern-aware outpainting. The input-specific
  adaptation phase involves conducting efficient and effective pseudo outpainting
  learning on the single-shot source video. This process encourages the model to
  identify and learn patterns within the source video, as well as bridging the
  gap between standard generative processes and outpainting. The subsequent
  phase, pattern-aware outpainting, is dedicated to the generalization of these
  learned patterns to generate outpainting outcomes. Additional strategies
  including spatial-aware insertion and noise travel are proposed to better
  leverage the diffusion model's generative prior and the acquired video patterns
  from source videos. Extensive evaluations underscore MOTIA's superiority,
  outperforming existing state-of-the-art methods in widely recognized
  benchmarks. Notably, these advancements are achieved without necessitating
  extensive, task-specific tuning.
paper_ai_keywords: diffusion-based pipeline input-specific adaptation pattern-aware outpainting pseudo outpainting spatial-aware insertion noise travel diffusion model's generative prior