Improving Robustness for Joint Optimization of Camera Poses and
  Decomposed Low-Rank Tensorial Radiance Fields
In this paper, we propose an algorithm that allows joint refinement of camera
pose and scene geometry represented by decomposed low-rank tensor, using only
2D images as supervision. First, we conduct a pilot study based on a 1D signal
and relate our findings to 3D scenarios, where the naive joint pose
optimization on voxel-based NeRFs can easily lead to sub-optimal solutions.
Moreover, based on the analysis of the frequency spectrum, we propose to apply
convolutional Gaussian filters on 2D and 3D radiance fields for a
coarse-to-fine training schedule that enables joint camera pose optimization.
Leveraging the decomposition property in decomposed low-rank tensor, our method
achieves an equivalent effect to brute-force 3D convolution with only incurring
little computational overhead. To further improve the robustness and stability
of joint optimization, we also propose techniques of smoothed 2D supervision,
randomly scaled kernel parameters, and edge-guided loss mask. Extensive
quantitative and qualitative evaluations demonstrate that our proposed
framework achieves superior performance in novel view synthesis as well as
rapid convergence for optimization.
65d575be6bf75d2c23989889
Bo-Yu Cheng
65d575be6bf75d2c2398988a
Wei-Chen Chiu
65d575be6bf75d2c2398988b
Yu-Lun Liu
admin_assigned
2024-02-21T10:16:44.020Z
6459d5da3b6fafd9664807ab
/avatars/57430d1bbde3a2fe5586e5fbcafb0e74.svg
Yu-Lun Liu
user
yulunliu
Improving Robustness for Joint Optimization of Camera Poses and
  Decomposed Low-Rank Tensorial Radiance Fields
In this paper, we propose an algorithm that allows joint refinement of camera
pose and scene geometry represented by decomposed low-rank tensor, using only
2D images as supervision. First, we conduct a pilot study based on a 1D signal
and relate our findings to 3D scenarios, where the naive joint pose
optimization on voxel-based NeRFs can easily lead to sub-optimal solutions.
Moreover, based on the analysis of the frequency spectrum, we propose to apply
convolutional Gaussian filters on 2D and 3D radiance fields for a
coarse-to-fine training schedule that enables joint camera pose optimization.
Leveraging the decomposition property in decomposed low-rank tensor, our method
achieves an equivalent effect to brute-force 3D convolution with only incurring
little computational overhead. To further improve the robustness and stability
of joint optimization, we also propose techniques of smoothed 2D supervision,
randomly scaled kernel parameters, and edge-guided loss mask. Extensive
quantitative and qualitative evaluations demonstrate that our proposed
framework achieves superior performance in novel view synthesis as well as
rapid convergence for optimization.
Proposed algorithm uses decomposed low-rank tensor and convolutional Gaussian filters for joint optimization of camera pose and scene geometry, improving performance in novel view synthesis and convergence speed.
low-rank tensor convolutional Gaussian filters NeRFs voxel-based frequency spectrum coarse-to-fine smoothed 2D supervision randomly scaled kernel parameters edge-guided loss mask joint optimization novel view synthesis rapid convergence