Learning Heterogeneous Mixture of Scene Experts for Large-scale Neural
  Radiance Fields
Recent NeRF methods on large-scale scenes have underlined the importance of
scene decomposition for scalable NeRFs. Although achieving reasonable
scalability, there are several critical problems remaining unexplored, i.e.,
learnable decomposition, modeling scene heterogeneity, and modeling efficiency.
In this paper, we introduce Switch-NeRF++, a Heterogeneous Mixture of Hash
Experts (HMoHE) network that addresses these challenges within a unified
framework. It is a highly scalable NeRF that learns heterogeneous decomposition
and heterogeneous NeRFs efficiently for large-scale scenes in an end-to-end
manner. In our framework, a gating network learns to decomposes scenes and
allocates 3D points to specialized NeRF experts. This gating network is
co-optimized with the experts, by our proposed Sparsely Gated Mixture of
Experts (MoE) NeRF framework. We incorporate a hash-based gating network and
distinct heterogeneous hash experts. The hash-based gating efficiently learns
the decomposition of the large-scale scene. The distinct heterogeneous hash
experts consist of hash grids of different resolution ranges, enabling
effective learning of the heterogeneous representation of different scene
parts. These design choices make our framework an end-to-end and highly
scalable NeRF solution for real-world large-scale scene modeling to achieve
both quality and efficiency. We evaluate our accuracy and scalability on
existing large-scale NeRF datasets and a new dataset with very large-scale
scenes (>6.5km^2) from UrbanBIS. Extensive experiments demonstrate that our
approach can be easily scaled to various large-scale scenes and achieve
state-of-the-art scene rendering accuracy. Furthermore, our method exhibits
significant efficiency, with an 8x acceleration in training and a 16x
acceleration in rendering compared to Switch-NeRF. Codes will be released in
https://github.com/MiZhenxing/Switch-NeRF.
6819f4557ef45d006f8b6755
Zhenxing Mi
claimed_verified
2025-05-06T16:51:34.076Z
6354bda206d707b33249c4c2
/avatars/bbd9f76274ac52214df92084d50bc7b5.svg
Zhenxing Mi
user
Mifucius
6819f4557ef45d006f8b6756
Ping Yin
6819f4557ef45d006f8b6757
Xue Xiao
6819f4557ef45d006f8b6758
Dan Xu
Learning Heterogeneous Mixture of Scene Experts for Large-scale Neural
  Radiance Fields
Recent NeRF methods on large-scale scenes have underlined the importance of
scene decomposition for scalable NeRFs. Although achieving reasonable
scalability, there are several critical problems remaining unexplored, i.e.,
learnable decomposition, modeling scene heterogeneity, and modeling efficiency.
In this paper, we introduce Switch-NeRF++, a Heterogeneous Mixture of Hash
Experts (HMoHE) network that addresses these challenges within a unified
framework. It is a highly scalable NeRF that learns heterogeneous decomposition
and heterogeneous NeRFs efficiently for large-scale scenes in an end-to-end
manner. In our framework, a gating network learns to decomposes scenes and
allocates 3D points to specialized NeRF experts. This gating network is
co-optimized with the experts, by our proposed Sparsely Gated Mixture of
Experts (MoE) NeRF framework. We incorporate a hash-based gating network and
distinct heterogeneous hash experts. The hash-based gating efficiently learns
the decomposition of the large-scale scene. The distinct heterogeneous hash
experts consist of hash grids of different resolution ranges, enabling
effective learning of the heterogeneous representation of different scene
parts. These design choices make our framework an end-to-end and highly
scalable NeRF solution for real-world large-scale scene modeling to achieve
both quality and efficiency. We evaluate our accuracy and scalability on
existing large-scale NeRF datasets and a new dataset with very large-scale
scenes (>6.5km^2) from UrbanBIS. Extensive experiments demonstrate that our
approach can be easily scaled to various large-scale scenes and achieve
state-of-the-art scene rendering accuracy. Furthermore, our method exhibits
significant efficiency, with an 8x acceleration in training and a 16x
acceleration in rendering compared to Switch-NeRF. Codes will be released in
https://github.com/MiZhenxing/Switch-NeRF.
Switch-NeRF++ is a scalable NeRF solution using a Heterogeneous Mixture of Hash Experts (HMoHE) network that addresses learnable decomposition, scene heterogeneity, and modeling efficiency.
Heterogeneous Mixture of Hash Experts (HMoHE) gating network Sparsely Gated Mixture of Experts (MoE) NeRF hash-based gating heterogeneous hash experts hash grids resolution ranges large-scale NeRF datasets UrbanBIS scene rendering accuracy training acceleration rendering acceleration