title: NVComposer: Boosting Generative Novel View Synthesis with Multiple
    Sparse and Unposed Images
summary: Recent advancements in generative models have significantly improved novel
  view synthesis (NVS) from multi-view data. However, existing methods depend on
  external multi-view alignment processes, such as explicit pose estimation or
  pre-reconstruction, which limits their flexibility and accessibility,
  especially when alignment is unstable due to insufficient overlap or occlusions
  between views. In this paper, we propose NVComposer, a novel approach that
  eliminates the need for explicit external alignment. NVComposer enables the
  generative model to implicitly infer spatial and geometric relationships
  between multiple conditional views by introducing two key components: 1) an
  image-pose dual-stream diffusion model that simultaneously generates target
  novel views and condition camera poses, and 2) a geometry-aware feature
  alignment module that distills geometric priors from dense stereo models during
  training. Extensive experiments demonstrate that NVComposer achieves
  state-of-the-art performance in generative multi-view NVS tasks, removing the
  reliance on external alignment and thus improving model accessibility. Our
  approach shows substantial improvements in synthesis quality as the number of
  unposed input views increases, highlighting its potential for more flexible and
  accessible generative NVS systems.
paper_authors._id: 675136f914bae121c8dcd5dc
paper_authors.name: Lingen Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-05T09:18:08.139Z
paper_authors._id: 66837d3c48edefb453b0640a
paper_authors.avatarUrl: /avatars/b16385eaa612578728e2c6460a76b38f.svg
paper_authors.fullname: Lingen Li
paper_authors.type: user
paper_authors.user: l-li
paper_authors._id: 675136f914bae121c8dcd5dd
paper_authors.name: Zhaoyang Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-05T09:18:10.618Z
paper_authors._id: 658409ceca19ccf6d9989add
paper_authors.avatarUrl: /avatars/3ac1dbd25e52435185babdeb3da28875.svg
paper_authors.fullname: Zhaoyang Zhang
paper_authors.type: user
paper_authors.user: ZyZcuhk
paper_authors._id: 675136f914bae121c8dcd5de
paper_authors.name: Yaowei Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-05T10:26:02.632Z
paper_authors._id: 6362801380c1a705a6ea54ac
paper_authors.avatarUrl: /avatars/041ad5abf9be42e336938f51ebb8746c.svg
paper_authors.fullname: Yaowei Li
paper_authors.type: user
paper_authors.user: Yw22
paper_authors._id: 675136f914bae121c8dcd5df
paper_authors.name: Jiale Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-05T10:26:54.054Z
paper_authors._id: 62c695829db11473f08af1cd
paper_authors.avatarUrl: /avatars/cacb54077892a44aef81454dc107df4f.svg
paper_authors.fullname: Jiale Xu
paper_authors.type: user
paper_authors.user: bluestyle97
paper_authors._id: 675136f914bae121c8dcd5e0
paper_authors.name: Xiaoyu Li
paper_authors._id: 675136f914bae121c8dcd5e1
paper_authors.name: Wenbo Hu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-10T08:04:44.511Z
paper_authors._id: 657a7458afbb0117ba15c59f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/657a7458afbb0117ba15c59f/8_iwTS1UG_mKnfylFbLsY.jpeg
paper_authors.fullname: Wenbo Hu
paper_authors.type: user
paper_authors.user: wbhu-tc
paper_authors._id: 675136f914bae121c8dcd5e2
paper_authors.name: Weihao Cheng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-26T11:39:44.903Z
paper_authors._id: 66052fd1dcacdaf58388502c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/66052fd1dcacdaf58388502c/RScS_YUFem34ArKEAjzX5.png
paper_authors.fullname: weihao
paper_authors.type: user
paper_authors.user: tokenid
paper_authors._id: 675136f914bae121c8dcd5e3
paper_authors.name: Jinwei Gu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-05T10:28:32.438Z
paper_authors._id: 647e8118770c299e56fc2bc8
paper_authors.avatarUrl: /avatars/adf80f3473dda42450148789ae5c208f.svg
paper_authors.fullname: Jinwei Gu
paper_authors.type: user
paper_authors.user: jwgu
paper_authors._id: 675136f914bae121c8dcd5e4
paper_authors.name: Tianfan Xue
paper_authors._id: 675136f914bae121c8dcd5e5
paper_authors.name: Ying Shan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-05T10:28:56.917Z
paper_authors._id: 63ca3ddc04c979828310bfcb
paper_authors.avatarUrl: /avatars/615e0d8622950b4408b40d550f02a894.svg
paper_authors.fullname: Ying Shan
paper_authors.type: user
paper_authors.user: yshan2u
paper_title: NVComposer: Boosting Generative Novel View Synthesis with Multiple
    Sparse and Unposed Images
paper_summary: Recent advancements in generative models have significantly improved novel
  view synthesis (NVS) from multi-view data. However, existing methods depend on
  external multi-view alignment processes, such as explicit pose estimation or
  pre-reconstruction, which limits their flexibility and accessibility,
  especially when alignment is unstable due to insufficient overlap or occlusions
  between views. In this paper, we propose NVComposer, a novel approach that
  eliminates the need for explicit external alignment. NVComposer enables the
  generative model to implicitly infer spatial and geometric relationships
  between multiple conditional views by introducing two key components: 1) an
  image-pose dual-stream diffusion model that simultaneously generates target
  novel views and condition camera poses, and 2) a geometry-aware feature
  alignment module that distills geometric priors from dense stereo models during
  training. Extensive experiments demonstrate that NVComposer achieves
  state-of-the-art performance in generative multi-view NVS tasks, removing the
  reliance on external alignment and thus improving model accessibility. Our
  approach shows substantial improvements in synthesis quality as the number of
  unposed input views increases, highlighting its potential for more flexible and
  accessible generative NVS systems.
paper_ai_keywords: dual-stream diffusion model image-pose condition camera poses geometry-aware feature alignment module dense stereo models generative multi-view NVS tasks synthesis quality