title: Improving Robustness for Joint Optimization of Camera Poses and
    Decomposed Low-Rank Tensorial Radiance Fields
summary: In this paper, we propose an algorithm that allows joint refinement of camera
  pose and scene geometry represented by decomposed low-rank tensor, using only
  2D images as supervision. First, we conduct a pilot study based on a 1D signal
  and relate our findings to 3D scenarios, where the naive joint pose
  optimization on voxel-based NeRFs can easily lead to sub-optimal solutions.
  Moreover, based on the analysis of the frequency spectrum, we propose to apply
  convolutional Gaussian filters on 2D and 3D radiance fields for a
  coarse-to-fine training schedule that enables joint camera pose optimization.
  Leveraging the decomposition property in decomposed low-rank tensor, our method
  achieves an equivalent effect to brute-force 3D convolution with only incurring
  little computational overhead. To further improve the robustness and stability
  of joint optimization, we also propose techniques of smoothed 2D supervision,
  randomly scaled kernel parameters, and edge-guided loss mask. Extensive
  quantitative and qualitative evaluations demonstrate that our proposed
  framework achieves superior performance in novel view synthesis as well as
  rapid convergence for optimization.
paper_authors._id: 65d575be6bf75d2c23989889
paper_authors.name: Bo-Yu Cheng
paper_authors._id: 65d575be6bf75d2c2398988a
paper_authors.name: Wei-Chen Chiu
paper_authors._id: 65d575be6bf75d2c2398988b
paper_authors.name: Yu-Lun Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-21T10:16:44.020Z
paper_authors._id: 6459d5da3b6fafd9664807ab
paper_authors.avatarUrl: /avatars/57430d1bbde3a2fe5586e5fbcafb0e74.svg
paper_authors.fullname: Yu-Lun Liu
paper_authors.type: user
paper_authors.user: yulunliu
paper_title: Improving Robustness for Joint Optimization of Camera Poses and
    Decomposed Low-Rank Tensorial Radiance Fields
paper_summary: In this paper, we propose an algorithm that allows joint refinement of camera
  pose and scene geometry represented by decomposed low-rank tensor, using only
  2D images as supervision. First, we conduct a pilot study based on a 1D signal
  and relate our findings to 3D scenarios, where the naive joint pose
  optimization on voxel-based NeRFs can easily lead to sub-optimal solutions.
  Moreover, based on the analysis of the frequency spectrum, we propose to apply
  convolutional Gaussian filters on 2D and 3D radiance fields for a
  coarse-to-fine training schedule that enables joint camera pose optimization.
  Leveraging the decomposition property in decomposed low-rank tensor, our method
  achieves an equivalent effect to brute-force 3D convolution with only incurring
  little computational overhead. To further improve the robustness and stability
  of joint optimization, we also propose techniques of smoothed 2D supervision,
  randomly scaled kernel parameters, and edge-guided loss mask. Extensive
  quantitative and qualitative evaluations demonstrate that our proposed
  framework achieves superior performance in novel view synthesis as well as
  rapid convergence for optimization.
paper_ai_keywords: low-rank tensor voxel-based NeRFs radiance fields convolutional Gaussian filters coarse-to-fine training schedule brute-force 3D convolution smoothed 2D supervision randomly scaled kernel parameters edge-guided loss mask novel view synthesis