title: Learning Heterogeneous Mixture of Scene Experts for Large-scale Neural
    Radiance Fields
summary: Recent NeRF methods on large-scale scenes have underlined the importance of
  scene decomposition for scalable NeRFs. Although achieving reasonable
  scalability, there are several critical problems remaining unexplored, i.e.,
  learnable decomposition, modeling scene heterogeneity, and modeling efficiency.
  In this paper, we introduce Switch-NeRF++, a Heterogeneous Mixture of Hash
  Experts (HMoHE) network that addresses these challenges within a unified
  framework. It is a highly scalable NeRF that learns heterogeneous decomposition
  and heterogeneous NeRFs efficiently for large-scale scenes in an end-to-end
  manner. In our framework, a gating network learns to decomposes scenes and
  allocates 3D points to specialized NeRF experts. This gating network is
  co-optimized with the experts, by our proposed Sparsely Gated Mixture of
  Experts (MoE) NeRF framework. We incorporate a hash-based gating network and
  distinct heterogeneous hash experts. The hash-based gating efficiently learns
  the decomposition of the large-scale scene. The distinct heterogeneous hash
  experts consist of hash grids of different resolution ranges, enabling
  effective learning of the heterogeneous representation of different scene
  parts. These design choices make our framework an end-to-end and highly
  scalable NeRF solution for real-world large-scale scene modeling to achieve
  both quality and efficiency. We evaluate our accuracy and scalability on
  existing large-scale NeRF datasets and a new dataset with very large-scale
  scenes (>6.5km^2) from UrbanBIS. Extensive experiments demonstrate that our
  approach can be easily scaled to various large-scale scenes and achieve
  state-of-the-art scene rendering accuracy. Furthermore, our method exhibits
  significant efficiency, with an 8x acceleration in training and a 16x
  acceleration in rendering compared to Switch-NeRF. Codes will be released in
  https://github.com/MiZhenxing/Switch-NeRF.
paper_authors._id: 6819f4557ef45d006f8b6755
paper_authors.name: Zhenxing Mi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-06T16:51:34.076Z
paper_authors._id: 6354bda206d707b33249c4c2
paper_authors.avatarUrl: /avatars/bbd9f76274ac52214df92084d50bc7b5.svg
paper_authors.fullname: Zhenxing Mi
paper_authors.type: user
paper_authors.user: Mifucius
paper_authors._id: 6819f4557ef45d006f8b6756
paper_authors.name: Ping Yin
paper_authors._id: 6819f4557ef45d006f8b6757
paper_authors.name: Xue Xiao
paper_authors._id: 6819f4557ef45d006f8b6758
paper_authors.name: Dan Xu
paper_title: Learning Heterogeneous Mixture of Scene Experts for Large-scale Neural
    Radiance Fields
paper_summary: Recent NeRF methods on large-scale scenes have underlined the importance of
  scene decomposition for scalable NeRFs. Although achieving reasonable
  scalability, there are several critical problems remaining unexplored, i.e.,
  learnable decomposition, modeling scene heterogeneity, and modeling efficiency.
  In this paper, we introduce Switch-NeRF++, a Heterogeneous Mixture of Hash
  Experts (HMoHE) network that addresses these challenges within a unified
  framework. It is a highly scalable NeRF that learns heterogeneous decomposition
  and heterogeneous NeRFs efficiently for large-scale scenes in an end-to-end
  manner. In our framework, a gating network learns to decomposes scenes and
  allocates 3D points to specialized NeRF experts. This gating network is
  co-optimized with the experts, by our proposed Sparsely Gated Mixture of
  Experts (MoE) NeRF framework. We incorporate a hash-based gating network and
  distinct heterogeneous hash experts. The hash-based gating efficiently learns
  the decomposition of the large-scale scene. The distinct heterogeneous hash
  experts consist of hash grids of different resolution ranges, enabling
  effective learning of the heterogeneous representation of different scene
  parts. These design choices make our framework an end-to-end and highly
  scalable NeRF solution for real-world large-scale scene modeling to achieve
  both quality and efficiency. We evaluate our accuracy and scalability on
  existing large-scale NeRF datasets and a new dataset with very large-scale
  scenes (>6.5km^2) from UrbanBIS. Extensive experiments demonstrate that our
  approach can be easily scaled to various large-scale scenes and achieve
  state-of-the-art scene rendering accuracy. Furthermore, our method exhibits
  significant efficiency, with an 8x acceleration in training and a 16x
  acceleration in rendering compared to Switch-NeRF. Codes will be released in
  https://github.com/MiZhenxing/Switch-NeRF.
paper_ai_summary: Switch-NeRF++ is a scalable NeRF solution using a Heterogeneous Mixture of Hash Experts (HMoHE) network that addresses learnable decomposition, scene heterogeneity, and modeling efficiency.
paper_ai_keywords: Heterogeneous Mixture of Hash Experts (HMoHE) gating network Sparsely Gated Mixture of Experts (MoE) NeRF hash-based gating heterogeneous hash experts hash grids resolution ranges large-scale NeRF datasets UrbanBIS scene rendering accuracy training acceleration rendering acceleration