Pippo: High-Resolution Multi-View Humans from a Single Image
We present Pippo, a generative model capable of producing 1K resolution dense
turnaround videos of a person from a single casually clicked photo. Pippo is a
multi-view diffusion transformer and does not require any additional inputs -
e.g., a fitted parametric model or camera parameters of the input image. We
pre-train Pippo on 3B human images without captions, and conduct multi-view
mid-training and post-training on studio captured humans. During mid-training,
to quickly absorb the studio dataset, we denoise several (up to 48) views at
low-resolution, and encode target cameras coarsely using a shallow MLP. During
post-training, we denoise fewer views at high-resolution and use pixel-aligned
controls (e.g., Spatial anchor and Plucker rays) to enable 3D consistent
generations. At inference, we propose an attention biasing technique that
allows Pippo to simultaneously generate greater than 5 times as many views as
seen during training. Finally, we also introduce an improved metric to evaluate
3D consistency of multi-view generations, and show that Pippo outperforms
existing works on multi-view human generation from a single image.
67aceb3315317375eccddc3b
Yash Kant
claimed_verified
2025-02-13T08:24:09.195Z
638546d0a179f856005ae310
/avatars/f7f2fface336c8e168a1daaf9fd4d40c.svg
yashkant
user
yashkant
67aceb3315317375eccddc3c
Ethan Weber
67aceb3315317375eccddc3d
Jin Kyu Kim
67aceb3315317375eccddc3e
Rawal Khirodkar
67aceb3315317375eccddc3f
Su Zhaoen
67aceb3315317375eccddc40
Julieta Martinez
67aceb3315317375eccddc41
Igor Gilitschenski
67aceb3315317375eccddc42
Shunsuke Saito
67aceb3315317375eccddc43
Timur Bagautdinov
Pippo: High-Resolution Multi-View Humans from a Single Image
We present Pippo, a generative model capable of producing 1K resolution dense
turnaround videos of a person from a single casually clicked photo. Pippo is a
multi-view diffusion transformer and does not require any additional inputs -
e.g., a fitted parametric model or camera parameters of the input image. We
pre-train Pippo on 3B human images without captions, and conduct multi-view
mid-training and post-training on studio captured humans. During mid-training,
to quickly absorb the studio dataset, we denoise several (up to 48) views at
low-resolution, and encode target cameras coarsely using a shallow MLP. During
post-training, we denoise fewer views at high-resolution and use pixel-aligned
controls (e.g., Spatial anchor and Plucker rays) to enable 3D consistent
generations. At inference, we propose an attention biasing technique that
allows Pippo to simultaneously generate greater than 5 times as many views as
seen during training. Finally, we also introduce an improved metric to evaluate
3D consistency of multi-view generations, and show that Pippo outperforms
existing works on multi-view human generation from a single image.
Pippo, a multi-view diffusion transformer model, generates high-resolution 3D consistent video turnarounds from a single photo using improved denoising techniques and attention biasing.
generative model diffusion transformer denoising multi-view camera parameters pre-training studio captured shallow MLP low-resolution high-resolution pixel-aligned controls Spatial anchor Plucker rays 3D consistent generations attention biasing 3D consistency metric