UnMix-NeRF: Spectral Unmixing Meets Neural Radiance Fields
Neural Radiance Field (NeRF)-based segmentation methods focus on object
semantics and rely solely on RGB data, lacking intrinsic material properties.
This limitation restricts accurate material perception, which is crucial for
robotics, augmented reality, simulation, and other applications. We introduce
UnMix-NeRF, a framework that integrates spectral unmixing into NeRF, enabling
joint hyperspectral novel view synthesis and unsupervised material
segmentation. Our method models spectral reflectance via diffuse and specular
components, where a learned dictionary of global endmembers represents pure
material signatures, and per-point abundances capture their distribution. For
material segmentation, we use spectral signature predictions along learned
endmembers, allowing unsupervised material clustering. Additionally, UnMix-NeRF
enables scene editing by modifying learned endmember dictionaries for flexible
material-based appearance manipulation. Extensive experiments validate our
approach, demonstrating superior spectral reconstruction and material
segmentation to existing methods. Project page:
https://www.factral.co/UnMix-NeRF.
686aa762213f123a1f88bcc4
Fabian Perez
claimed_verified
2025-07-09T08:51:40.001Z
62eee05235b4995510af75bf
https://cdn-avatars.huggingface.co/v1/production/uploads/1659822155165-noauth.jpeg
Fabian Perez
user
Factral
686aa762213f123a1f88bcc5
Sara Rojas
686aa762213f123a1f88bcc6
Carlos Hinojosa
claimed_verified
2025-07-15T19:11:29.724Z
6620e21d11561bf979229d9f
https://cdn-avatars.huggingface.co/v1/production/uploads/6620e21d11561bf979229d9f/qzMMJI4PJfEkJDYK4mWZM.jpeg
Carlos Hinojosa
user
carlosh93
686aa762213f123a1f88bcc7
Hoover Rueda-Chacón
686aa762213f123a1f88bcc8
Bernard Ghanem
UnMix-NeRF: Spectral Unmixing Meets Neural Radiance Fields
Neural Radiance Field (NeRF)-based segmentation methods focus on object
semantics and rely solely on RGB data, lacking intrinsic material properties.
This limitation restricts accurate material perception, which is crucial for
robotics, augmented reality, simulation, and other applications. We introduce
UnMix-NeRF, a framework that integrates spectral unmixing into NeRF, enabling
joint hyperspectral novel view synthesis and unsupervised material
segmentation. Our method models spectral reflectance via diffuse and specular
components, where a learned dictionary of global endmembers represents pure
material signatures, and per-point abundances capture their distribution. For
material segmentation, we use spectral signature predictions along learned
endmembers, allowing unsupervised material clustering. Additionally, UnMix-NeRF
enables scene editing by modifying learned endmember dictionaries for flexible
material-based appearance manipulation. Extensive experiments validate our
approach, demonstrating superior spectral reconstruction and material
segmentation to existing methods. Project page:
https://www.factral.co/UnMix-NeRF.
UnMix-NeRF integrates spectral unmixing into NeRF for joint hyperspectral novel view synthesis and unsupervised material segmentation, enhancing material perception and scene editing capabilities.
Neural Radiance Field NeRF spectral unmixing hyperspectral novel view synthesis unsupervised material segmentation spectral reflectance diffuse components specular components global endmembers per-point abundances material clustering scene editing material-based appearance manipulation