Dens3R: A Foundation Model for 3D Geometry Prediction
Recent advances in dense 3D reconstruction have led to significant progress,
yet achieving accurate unified geometric prediction remains a major challenge.
Most existing methods are limited to predicting a single geometry quantity from
input images. However, geometric quantities such as depth, surface normals, and
point maps are inherently correlated, and estimating them in isolation often
fails to ensure consistency, thereby limiting both accuracy and practical
applicability. This motivates us to explore a unified framework that explicitly
models the structural coupling among different geometric properties to enable
joint regression. In this paper, we present Dens3R, a 3D foundation model
designed for joint geometric dense prediction and adaptable to a wide range of
downstream tasks. Dens3R adopts a two-stage training framework to progressively
build a pointmap representation that is both generalizable and intrinsically
invariant. Specifically, we design a lightweight shared encoder-decoder
backbone and introduce position-interpolated rotary positional encoding to
maintain expressive power while enhancing robustness to high-resolution inputs.
By integrating image-pair matching features with intrinsic invariance modeling,
Dens3R accurately regresses multiple geometric quantities such as surface
normals and depth, achieving consistent geometry perception from single-view to
multi-view inputs. Additionally, we propose a post-processing pipeline that
supports geometrically consistent multi-view inference. Extensive experiments
demonstrate the superior performance of Dens3R across various dense 3D
prediction tasks and highlight its potential for broader applications.
689225269dcf0e7ac9a66eb7
Xianze Fang
689225269dcf0e7ac9a66eb8
Jingnan Gao
689225269dcf0e7ac9a66eb9
Zhe Wang
689225269dcf0e7ac9a66eba
Zhuo Chen
689225269dcf0e7ac9a66ebb
Xingyu Ren
689225269dcf0e7ac9a66ebc
Jiangjing Lyu
689225269dcf0e7ac9a66ebd
Qiaomu Ren
689225269dcf0e7ac9a66ebe
Zhonglei Yang
689225269dcf0e7ac9a66ebf
Xiaokang Yang
689225269dcf0e7ac9a66ec0
Yichao Yan
689225269dcf0e7ac9a66ec1
Chengfei Lyu
Dens3R: A Foundation Model for 3D Geometry Prediction
Recent advances in dense 3D reconstruction have led to significant progress,
yet achieving accurate unified geometric prediction remains a major challenge.
Most existing methods are limited to predicting a single geometry quantity from
input images. However, geometric quantities such as depth, surface normals, and
point maps are inherently correlated, and estimating them in isolation often
fails to ensure consistency, thereby limiting both accuracy and practical
applicability. This motivates us to explore a unified framework that explicitly
models the structural coupling among different geometric properties to enable
joint regression. In this paper, we present Dens3R, a 3D foundation model
designed for joint geometric dense prediction and adaptable to a wide range of
downstream tasks. Dens3R adopts a two-stage training framework to progressively
build a pointmap representation that is both generalizable and intrinsically
invariant. Specifically, we design a lightweight shared encoder-decoder
backbone and introduce position-interpolated rotary positional encoding to
maintain expressive power while enhancing robustness to high-resolution inputs.
By integrating image-pair matching features with intrinsic invariance modeling,
Dens3R accurately regresses multiple geometric quantities such as surface
normals and depth, achieving consistent geometry perception from single-view to
multi-view inputs. Additionally, we propose a post-processing pipeline that
supports geometrically consistent multi-view inference. Extensive experiments
demonstrate the superior performance of Dens3R across various dense 3D
prediction tasks and highlight its potential for broader applications.
Dens3R is a 3D foundation model that jointly predicts multiple geometric quantities using a two-stage training framework, enhancing consistency and performance in dense 3D reconstruction tasks.
3D reconstruction dense 3D prediction geometric quantities depth surface normals point maps unified framework structural coupling joint regression 3D foundation model two-stage training pointmap representation lightweight shared encoder-decoder position-interpolated rotary positional encoding image-pair matching intrinsic invariance multi-view inference