Acoustic Volume Rendering for Neural Impulse Response Fields
Realistic audio synthesis that captures accurate acoustic phenomena is
essential for creating immersive experiences in virtual and augmented reality.
Synthesizing the sound received at any position relies on the estimation of
impulse response (IR), which characterizes how sound propagates in one scene
along different paths before arriving at the listener's position. In this
paper, we present Acoustic Volume Rendering (AVR), a novel approach that adapts
volume rendering techniques to model acoustic impulse responses. While volume
rendering has been successful in modeling radiance fields for images and neural
scene representations, IRs present unique challenges as time-series signals. To
address these challenges, we introduce frequency-domain volume rendering and
use spherical integration to fit the IR measurements. Our method constructs an
impulse response field that inherently encodes wave propagation principles and
achieves state-of-the-art performance in synthesizing impulse responses for
novel poses. Experiments show that AVR surpasses current leading methods by a
substantial margin. Additionally, we develop an acoustic simulation platform,
AcoustiX, which provides more accurate and realistic IR simulations than
existing simulators. Code for AVR and AcoustiX are available at
https://zitonglan.github.io/avr.
6732bbb96c671fb849ad965a
Zitong Lan
6732bbb96c671fb849ad965b
Chenhao Zheng
6732bbb96c671fb849ad965c
Zhiwei Zheng
admin_assigned
2024-11-13T16:56:01.507Z
6340e9c67158d417a0fdb0c5
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/OeRx05yiLURO_uUx0AYux.png
Zhiwei
user
zzwalala
6732bbb96c671fb849ad965d
Mingmin Zhao
Acoustic Volume Rendering for Neural Impulse Response Fields
Realistic audio synthesis that captures accurate acoustic phenomena is
essential for creating immersive experiences in virtual and augmented reality.
Synthesizing the sound received at any position relies on the estimation of
impulse response (IR), which characterizes how sound propagates in one scene
along different paths before arriving at the listener's position. In this
paper, we present Acoustic Volume Rendering (AVR), a novel approach that adapts
volume rendering techniques to model acoustic impulse responses. While volume
rendering has been successful in modeling radiance fields for images and neural
scene representations, IRs present unique challenges as time-series signals. To
address these challenges, we introduce frequency-domain volume rendering and
use spherical integration to fit the IR measurements. Our method constructs an
impulse response field that inherently encodes wave propagation principles and
achieves state-of-the-art performance in synthesizing impulse responses for
novel poses. Experiments show that AVR surpasses current leading methods by a
substantial margin. Additionally, we develop an acoustic simulation platform,
AcoustiX, which provides more accurate and realistic IR simulations than
existing simulators. Code for AVR and AcoustiX are available at
https://zitonglan.github.io/avr.
Acoustic Volume Rendering (AVR) uses frequency-domain volume rendering and spherical integration to synthesize impulse responses more accurately and realistically than leading methods.
acoustic volume rendering impulse response (IR) volume rendering frequency-domain volume rendering spherical integration impulse response field wave propagation principles acoustic simulation platform