Diffuman4D: 4D Consistent Human View Synthesis from Sparse-View Videos
  with Spatio-Temporal Diffusion Models
This paper addresses the challenge of high-fidelity view synthesis of humans
with sparse-view videos as input. Previous methods solve the issue of
insufficient observation by leveraging 4D diffusion models to generate videos
at novel viewpoints. However, the generated videos from these models often lack
spatio-temporal consistency, thus degrading view synthesis quality. In this
paper, we propose a novel sliding iterative denoising process to enhance the
spatio-temporal consistency of the 4D diffusion model. Specifically, we define
a latent grid in which each latent encodes the image, camera pose, and human
pose for a certain viewpoint and timestamp, then alternately denoising the
latent grid along spatial and temporal dimensions with a sliding window, and
finally decode the videos at target viewpoints from the corresponding denoised
latents. Through the iterative sliding, information flows sufficiently across
the latent grid, allowing the diffusion model to obtain a large receptive field
and thus enhance the 4D consistency of the output, while making the GPU memory
consumption affordable. The experiments on the DNA-Rendering and ActorsHQ
datasets demonstrate that our method is able to synthesize high-quality and
consistent novel-view videos and significantly outperforms the existing
approaches. See our project page for interactive demos and video results:
https://diffuman4d.github.io/ .
6879f3aa21b37e676c8e4202
Yudong Jin
claimed_verified
2025-07-18T07:49:22.520Z
649958942ca6f96c8b8c1076
https://cdn-avatars.huggingface.co/v1/production/uploads/649958942ca6f96c8b8c1076/olfLIqNryaog1nAnQPwkN.jpeg
Yudong Jin
user
krahets
6879f3aa21b37e676c8e4203
Sida Peng
6879f3aa21b37e676c8e4204
Xuan Wang
6879f3aa21b37e676c8e4205
Tao Xie
6879f3aa21b37e676c8e4206
Zhen Xu
6879f3aa21b37e676c8e4207
Yifan Yang
6879f3aa21b37e676c8e4208
Yujun Shen
6879f3aa21b37e676c8e4209
Hujun Bao
6879f3aa21b37e676c8e420a
Xiaowei Zhou
Diffuman4D: 4D Consistent Human View Synthesis from Sparse-View Videos
  with Spatio-Temporal Diffusion Models
This paper addresses the challenge of high-fidelity view synthesis of humans
with sparse-view videos as input. Previous methods solve the issue of
insufficient observation by leveraging 4D diffusion models to generate videos
at novel viewpoints. However, the generated videos from these models often lack
spatio-temporal consistency, thus degrading view synthesis quality. In this
paper, we propose a novel sliding iterative denoising process to enhance the
spatio-temporal consistency of the 4D diffusion model. Specifically, we define
a latent grid in which each latent encodes the image, camera pose, and human
pose for a certain viewpoint and timestamp, then alternately denoising the
latent grid along spatial and temporal dimensions with a sliding window, and
finally decode the videos at target viewpoints from the corresponding denoised
latents. Through the iterative sliding, information flows sufficiently across
the latent grid, allowing the diffusion model to obtain a large receptive field
and thus enhance the 4D consistency of the output, while making the GPU memory
consumption affordable. The experiments on the DNA-Rendering and ActorsHQ
datasets demonstrate that our method is able to synthesize high-quality and
consistent novel-view videos and significantly outperforms the existing
approaches. See our project page for interactive demos and video results:
https://diffuman4d.github.io/ .
A sliding iterative denoising process enhances spatio-temporal consistency in 4D diffusion models for high-fidelity view synthesis from sparse-view videos.
4D diffusion models sliding iterative denoising latent grid image camera pose human pose spatio-temporal consistency GPU memory consumption DNA-Rendering ActorsHQ