title: SonicSim: A customizable simulation platform for speech processing in
    moving sound source scenarios
summary: The systematic evaluation of speech separation and enhancement models under
  moving sound source conditions typically requires extensive data comprising
  diverse scenarios. However, real-world datasets often contain insufficient data
  to meet the training and evaluation requirements of models. Although synthetic
  datasets offer a larger volume of data, their acoustic simulations lack
  realism. Consequently, neither real-world nor synthetic datasets effectively
  fulfill practical needs. To address these issues, we introduce SonicSim, a
  synthetic toolkit de-designed to generate highly customizable data for moving
  sound sources. SonicSim is developed based on the embodied AI simulation
  platform, Habitat-sim, supporting multi-level adjustments, including
  scene-level, microphone-level, and source-level, thereby generating more
  diverse synthetic data. Leveraging SonicSim, we constructed a moving sound
  source benchmark dataset, SonicSet, using the Librispeech, the Freesound
  Dataset 50k (FSD50K) and Free Music Archive (FMA), and 90 scenes from the
  Matterport3D to evaluate speech separation and enhancement models.
  Additionally, to validate the differences between synthetic data and real-world
  data, we randomly selected 5 hours of raw data without reverberation from the
  SonicSet validation set to record a real-world speech separation dataset, which
  was then compared with the corresponding synthetic datasets. Similarly, we
  utilized the real-world speech enhancement dataset RealMAN to validate the
  acoustic gap between other synthetic datasets and the SonicSet dataset for
  speech enhancement. The results indicate that the synthetic data generated by
  SonicSim can effectively generalize to real-world scenarios. Demo and code are
  publicly available at https://cslikai.cn/SonicSim/.
paper_authors._id: 66feb6df001816b29e69cd78
paper_authors.name: Kai Li
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-10-03T15:23:13.186Z
paper_authors._id: 6387676c23da90491eb9fb16
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1669818175965-noauth.jpeg
paper_authors.fullname: Kai Li
paper_authors.type: user
paper_authors.user: JusperLee
paper_authors._id: 66feb6df001816b29e69cd79
paper_authors.name: Wendi Sang
paper_authors._id: 66feb6df001816b29e69cd7a
paper_authors.name: Chang Zeng
paper_authors._id: 66feb6df001816b29e69cd7b
paper_authors.name: Runxuan Yang
paper_authors._id: 66feb6df001816b29e69cd7c
paper_authors.name: Guo Chen
paper_authors._id: 66feb6df001816b29e69cd7d
paper_authors.name: Xiaolin Hu
paper_title: SonicSim: A customizable simulation platform for speech processing in
    moving sound source scenarios
paper_summary: The systematic evaluation of speech separation and enhancement models under
  moving sound source conditions typically requires extensive data comprising
  diverse scenarios. However, real-world datasets often contain insufficient data
  to meet the training and evaluation requirements of models. Although synthetic
  datasets offer a larger volume of data, their acoustic simulations lack
  realism. Consequently, neither real-world nor synthetic datasets effectively
  fulfill practical needs. To address these issues, we introduce SonicSim, a
  synthetic toolkit de-designed to generate highly customizable data for moving
  sound sources. SonicSim is developed based on the embodied AI simulation
  platform, Habitat-sim, supporting multi-level adjustments, including
  scene-level, microphone-level, and source-level, thereby generating more
  diverse synthetic data. Leveraging SonicSim, we constructed a moving sound
  source benchmark dataset, SonicSet, using the Librispeech, the Freesound
  Dataset 50k (FSD50K) and Free Music Archive (FMA), and 90 scenes from the
  Matterport3D to evaluate speech separation and enhancement models.
  Additionally, to validate the differences between synthetic data and real-world
  data, we randomly selected 5 hours of raw data without reverberation from the
  SonicSet validation set to record a real-world speech separation dataset, which
  was then compared with the corresponding synthetic datasets. Similarly, we
  utilized the real-world speech enhancement dataset RealMAN to validate the
  acoustic gap between other synthetic datasets and the SonicSet dataset for
  speech enhancement. The results indicate that the synthetic data generated by
  SonicSim can effectively generalize to real-world scenarios. Demo and code are
  publicly available at https://cslikai.cn/SonicSim/.
paper_ai_keywords: SonicSim Habitat-sim scene-level microphone-level source-level SonicSet Librispeech Freesound Dataset 50k (FSD50K) Free Music Archive (FMA) Matterport3D RealMAN