NVComposer: Boosting Generative Novel View Synthesis with Multiple
  Sparse and Unposed Images
Recent advancements in generative models have significantly improved novel
view synthesis (NVS) from multi-view data. However, existing methods depend on
external multi-view alignment processes, such as explicit pose estimation or
pre-reconstruction, which limits their flexibility and accessibility,
especially when alignment is unstable due to insufficient overlap or occlusions
between views. In this paper, we propose NVComposer, a novel approach that
eliminates the need for explicit external alignment. NVComposer enables the
generative model to implicitly infer spatial and geometric relationships
between multiple conditional views by introducing two key components: 1) an
image-pose dual-stream diffusion model that simultaneously generates target
novel views and condition camera poses, and 2) a geometry-aware feature
alignment module that distills geometric priors from dense stereo models during
training. Extensive experiments demonstrate that NVComposer achieves
state-of-the-art performance in generative multi-view NVS tasks, removing the
reliance on external alignment and thus improving model accessibility. Our
approach shows substantial improvements in synthesis quality as the number of
unposed input views increases, highlighting its potential for more flexible and
accessible generative NVS systems.
675136f914bae121c8dcd5dc
Lingen Li
claimed_verified
2024-12-05T09:18:08.139Z
66837d3c48edefb453b0640a
/avatars/b16385eaa612578728e2c6460a76b38f.svg
Lingen Li
user
l-li
675136f914bae121c8dcd5dd
Zhaoyang Zhang
claimed_verified
2024-12-05T09:18:10.618Z
658409ceca19ccf6d9989add
/avatars/3ac1dbd25e52435185babdeb3da28875.svg
Zhaoyang Zhang
user
ZyZcuhk
675136f914bae121c8dcd5de
Yaowei Li
admin_assigned
2024-12-05T10:26:02.632Z
6362801380c1a705a6ea54ac
/avatars/041ad5abf9be42e336938f51ebb8746c.svg
Yaowei Li
user
Yw22
675136f914bae121c8dcd5df
Jiale Xu
admin_assigned
2024-12-05T10:26:54.054Z
62c695829db11473f08af1cd
/avatars/cacb54077892a44aef81454dc107df4f.svg
Jiale Xu
user
bluestyle97
675136f914bae121c8dcd5e0
Xiaoyu Li
675136f914bae121c8dcd5e1
Wenbo Hu
claimed_verified
2025-03-10T08:04:44.511Z
657a7458afbb0117ba15c59f
https://cdn-avatars.huggingface.co/v1/production/uploads/657a7458afbb0117ba15c59f/8_iwTS1UG_mKnfylFbLsY.jpeg
Wenbo Hu
user
wbhu-tc
675136f914bae121c8dcd5e2
Weihao Cheng
claimed_verified
2025-01-26T11:39:44.903Z
66052fd1dcacdaf58388502c
https://cdn-avatars.huggingface.co/v1/production/uploads/66052fd1dcacdaf58388502c/RScS_YUFem34ArKEAjzX5.png
weihao
user
tokenid
675136f914bae121c8dcd5e3
Jinwei Gu
admin_assigned
2024-12-05T10:28:32.438Z
647e8118770c299e56fc2bc8
/avatars/adf80f3473dda42450148789ae5c208f.svg
Jinwei Gu
user
jwgu
675136f914bae121c8dcd5e4
Tianfan Xue
675136f914bae121c8dcd5e5
Ying Shan
admin_assigned
2024-12-05T10:28:56.917Z
63ca3ddc04c979828310bfcb
/avatars/615e0d8622950b4408b40d550f02a894.svg
Ying Shan
user
yshan2u
NVComposer: Boosting Generative Novel View Synthesis with Multiple
  Sparse and Unposed Images
Recent advancements in generative models have significantly improved novel
view synthesis (NVS) from multi-view data. However, existing methods depend on
external multi-view alignment processes, such as explicit pose estimation or
pre-reconstruction, which limits their flexibility and accessibility,
especially when alignment is unstable due to insufficient overlap or occlusions
between views. In this paper, we propose NVComposer, a novel approach that
eliminates the need for explicit external alignment. NVComposer enables the
generative model to implicitly infer spatial and geometric relationships
between multiple conditional views by introducing two key components: 1) an
image-pose dual-stream diffusion model that simultaneously generates target
novel views and condition camera poses, and 2) a geometry-aware feature
alignment module that distills geometric priors from dense stereo models during
training. Extensive experiments demonstrate that NVComposer achieves
state-of-the-art performance in generative multi-view NVS tasks, removing the
reliance on external alignment and thus improving model accessibility. Our
approach shows substantial improvements in synthesis quality as the number of
unposed input views increases, highlighting its potential for more flexible and
accessible generative NVS systems.
NVComposer, a novel generative approach, achieves state-of-the-art performance in novel view synthesis by implicitly inferring spatial and geometric relationships between multiple views without requiring explicit external alignment.
generative models novel view synthesis multi-view alignment image-pose dual-stream diffusion model geometry-aware feature alignment module dense stereo models