title: Improving Inference-Time Optimisation for Vocal Effects Style Transfer
    with a Gaussian Prior
summary: Style Transfer with Inference-Time Optimisation (ST-ITO) is a recent approach
  for transferring the applied effects of a reference audio to a raw audio track.
  It optimises the effect parameters to minimise the distance between the style
  embeddings of the processed audio and the reference. However, this method
  treats all possible configurations equally and relies solely on the embedding
  space, which can lead to unrealistic or biased results. We address this pitfall
  by introducing a Gaussian prior derived from a vocal preset dataset, DiffVox,
  over the parameter space. The resulting optimisation is equivalent to
  maximum-a-posteriori estimation. Evaluations on vocal effects transfer on the
  MedleyDB dataset show significant improvements across metrics compared to
  baselines, including a blind audio effects estimator, nearest-neighbour
  approaches, and uncalibrated ST-ITO. The proposed calibration reduces parameter
  mean squared error by up to 33% and matches the reference style better.
  Subjective evaluations with 16 participants confirm our method's superiority,
  especially in limited data regimes. This work demonstrates how incorporating
  prior knowledge in inference time enhances audio effects transfer, paving the
  way for more effective and realistic audio processing systems.
paper_authors._id: 682b539908b49a9e5088330f
paper_authors.name: Chin-Yun Yu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-20T07:22:12.615Z
paper_authors._id: 621d85a10e35b2fbbf3e6196
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/621d85a10e35b2fbbf3e6196/D6DMntYh-eV5fVef_Mx4Z.png
paper_authors.fullname: Chin-Yun Yu
paper_authors.type: user
paper_authors.user: yoyolicoris
paper_authors._id: 682b539908b49a9e50883310
paper_authors.name: Marco A. Martínez-Ramírez
paper_authors._id: 682b539908b49a9e50883311
paper_authors.name: Junghyun Koo
paper_authors._id: 682b539908b49a9e50883312
paper_authors.name: Wei-Hsiang Liao
paper_authors._id: 682b539908b49a9e50883313
paper_authors.name: Yuki Mitsufuji
paper_authors._id: 682b539908b49a9e50883314
paper_authors.name: György Fazekas
paper_title: Improving Inference-Time Optimisation for Vocal Effects Style Transfer
    with a Gaussian Prior
paper_summary: Style Transfer with Inference-Time Optimisation (ST-ITO) is a recent approach
  for transferring the applied effects of a reference audio to a raw audio track.
  It optimises the effect parameters to minimise the distance between the style
  embeddings of the processed audio and the reference. However, this method
  treats all possible configurations equally and relies solely on the embedding
  space, which can lead to unrealistic or biased results. We address this pitfall
  by introducing a Gaussian prior derived from a vocal preset dataset, DiffVox,
  over the parameter space. The resulting optimisation is equivalent to
  maximum-a-posteriori estimation. Evaluations on vocal effects transfer on the
  MedleyDB dataset show significant improvements across metrics compared to
  baselines, including a blind audio effects estimator, nearest-neighbour
  approaches, and uncalibrated ST-ITO. The proposed calibration reduces parameter
  mean squared error by up to 33% and matches the reference style better.
  Subjective evaluations with 16 participants confirm our method's superiority,
  especially in limited data regimes. This work demonstrates how incorporating
  prior knowledge in inference time enhances audio effects transfer, paving the
  way for more effective and realistic audio processing systems.
paper_ai_summary: Incorporating Gaussian prior knowledge derived from a vocal preset dataset enhances audio effects transfer by improving parameter accuracy and matching reference styles more effectively than existing methods.
paper_ai_keywords: Gaussian prior DiffVox maximum-a-posteriori estimation MedleyDB audio effects transfer parameter mean squared error