Facing the Music: Tackling Singing Voice Separation in Cinematic Audio
  Source Separation
Cinematic audio source separation (CASS) is a fairly new subtask of audio
source separation. A typical setup of CASS is a three-stem problem, with the
aim of separating the mixture into the dialogue stem (DX), music stem (MX), and
effects stem (FX). In practice, however, several edge cases exist as some sound
sources do not fit neatly in either of these three stems, necessitating the use
of additional auxiliary stems in production. One very common edge case is the
singing voice in film audio, which may belong in either the DX or MX, depending
heavily on the cinematic context. In this work, we demonstrate a very
straightforward extension of the dedicated-decoder Bandit and query-based
single-decoder Banquet models to a four-stem problem, treating non-musical
dialogue, instrumental music, singing voice, and effects as separate stems.
Interestingly, the query-based Banquet model outperformed the dedicated-decoder
Bandit model. We hypothesized that this is due to a better feature alignment at
the bottleneck as enforced by the band-agnostic FiLM layer. Dataset and model
implementation will be made available at
https://github.com/kwatcharasupat/source-separation-landing.
66b4369e8a1be1e8acc7fb17
Karn N. Watcharasupat
extracted_confirmed
2024-08-08T18:33:31.743Z
66473a146fa6cf6e541d946d
/avatars/ba8948fbaecc89d7d8ef59416cf9eb9d.svg
Karn Watcharasupat
user
kwatcharasupat
66b4369e8a1be1e8acc7fb18
Chih-Wei Wu
admin_assigned
2024-08-08T10:50:32.609Z
664be5ee0f633df0c13e78d3
/avatars/f8c8add031febe32212717fc7dfeae5f.svg
Chih-Wei Wu
user
cwu307
66b4369e8a1be1e8acc7fb19
Iroro Orife
Facing the Music: Tackling Singing Voice Separation in Cinematic Audio
  Source Separation
Cinematic audio source separation (CASS) is a fairly new subtask of audio
source separation. A typical setup of CASS is a three-stem problem, with the
aim of separating the mixture into the dialogue stem (DX), music stem (MX), and
effects stem (FX). In practice, however, several edge cases exist as some sound
sources do not fit neatly in either of these three stems, necessitating the use
of additional auxiliary stems in production. One very common edge case is the
singing voice in film audio, which may belong in either the DX or MX, depending
heavily on the cinematic context. In this work, we demonstrate a very
straightforward extension of the dedicated-decoder Bandit and query-based
single-decoder Banquet models to a four-stem problem, treating non-musical
dialogue, instrumental music, singing voice, and effects as separate stems.
Interestingly, the query-based Banquet model outperformed the dedicated-decoder
Bandit model. We hypothesized that this is due to a better feature alignment at
the bottleneck as enforced by the band-agnostic FiLM layer. Dataset and model
implementation will be made available at
https://github.com/kwatcharasupat/source-separation-landing.
The query-based Banquet model outperforms the dedicated-decoder Bandit model in a four-stem Cinematic Audio Source Separation task by better aligning features through a band-agnostic FiLM layer.
Bandit Banquet FiLM layer Cinematic Audio Source Separation CASS four-stem problem dialogue stem music stem effects stem singing voice