Revisiting Depth Representations for Feed-Forward 3D Gaussian Splatting
Depth maps are widely used in feed-forward 3D Gaussian Splatting (3DGS)
pipelines by unprojecting them into 3D point clouds for novel view synthesis.
This approach offers advantages such as efficient training, the use of known
camera poses, and accurate geometry estimation. However, depth discontinuities
at object boundaries often lead to fragmented or sparse point clouds, degrading
rendering quality -- a well-known limitation of depth-based representations. To
tackle this issue, we introduce PM-Loss, a novel regularization loss based on a
pointmap predicted by a pre-trained transformer. Although the pointmap itself
may be less accurate than the depth map, it effectively enforces geometric
smoothness, especially around object boundaries. With the improved depth map,
our method significantly improves the feed-forward 3DGS across various
architectures and scenes, delivering consistently better rendering results. Our
project page: https://aim-uofa.github.io/PMLoss
6842591962047f5641b3b650
Duochao Shi
claimed_verified
2025-06-06T07:40:58.621Z
661d1f83ea3df2195a7c2924
/avatars/dec49fc1d79913b07b57ccbef079198f.svg
dcshi
user
dc-walker
6842591962047f5641b3b651
Weijie Wang
claimed_verified
2025-06-06T07:41:00.966Z
66699aa8a33847217b5a49c7
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/u8Z-6U8U7ARXOpdBDI7Qm.png
Weijie Wang
user
lhmd
6842591962047f5641b3b652
Donny Y. Chen
6842591962047f5641b3b653
Zeyu Zhang
6842591962047f5641b3b654
Jia-Wang Bian
6842591962047f5641b3b655
Bohan Zhuang
6842591962047f5641b3b656
Chunhua Shen
Revisiting Depth Representations for Feed-Forward 3D Gaussian Splatting
Depth maps are widely used in feed-forward 3D Gaussian Splatting (3DGS)
pipelines by unprojecting them into 3D point clouds for novel view synthesis.
This approach offers advantages such as efficient training, the use of known
camera poses, and accurate geometry estimation. However, depth discontinuities
at object boundaries often lead to fragmented or sparse point clouds, degrading
rendering quality -- a well-known limitation of depth-based representations. To
tackle this issue, we introduce PM-Loss, a novel regularization loss based on a
pointmap predicted by a pre-trained transformer. Although the pointmap itself
may be less accurate than the depth map, it effectively enforces geometric
smoothness, especially around object boundaries. With the improved depth map,
our method significantly improves the feed-forward 3DGS across various
architectures and scenes, delivering consistently better rendering results. Our
project page: https://aim-uofa.github.io/PMLoss
PM-Loss, a regularization technique using pointmaps from a pre-trained transformer, enhances feed-forward 3D Gaussian Splatting by improving depth map accuracy and rendering quality.
3D Gaussian Splatting 3DGS depth maps point clouds novel view synthesis PM-Loss pre-trained transformer pointmap geometric smoothness