Answers.Body: <p>The original wavenet and the implementation you linked to is globally conditioned on speaker embeddings, which means that the network was given a unique identifier for the person speaking each time it trained on an audio clip. This allows the network to learn to mimic the voice of each person in the training data, but it only learns <em>their</em> voices, not arbitrary people's voices.</p>
  <p>You might be able to do what you're describing by globally conditioning on different features of the speakers, such as age etc, though this would require a lot a speakers to span those dimensions.</p>
  <p>I think it would definitely be possible to use style transfer for this, similar to the work done on tweaking people's age in images [1]. Some similar work has been done for audio in order to change the accent of a voices between accents [2], so I wouldn't be surprised to see more examples of this in the near future.</p>
  <p>[1] <a href="https://www.youtube.com/watch?v=2qMw8sOsNg0" rel="nofollow noreferrer">https://www.youtube.com/watch?v=2qMw8sOsNg0</a>
  [2] <a href="https://ai.googleblog.com/2018/03/expressive-speech-synthesis-with.html" rel="nofollow noreferrer">https://ai.googleblog.com/2018/03/expressive-speech-synthesis-with.html</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-10-31T02:20:28.127Z
Answers.LastActivityDate: 2020-10-31T02:20:28.127Z
Answers.UserDisplayName: Robz
Body: <p>If one uses one of the open source implementations of the <a href="https://deepmind.com/blog/wavenet-generative-model-raw-audio/" rel="nofollow noreferrer">WaveNet generative speech synthesis design</a>, such as <a href="https://r9y9.github.io/wavenet_vocoder/" rel="nofollow noreferrer">https://r9y9.github.io/wavenet_vocoder/</a>, and trains using something like the <a href="http://festvox.org/cmu_arctic/" rel="nofollow noreferrer">CMU's arctic corpus</a>, now can one add a voice that sounds younger, older, less professional, or in some other way distinctive.  Must the entire training begin from scratch, or is there a more resource and time friendly way?</p>
  
Tags: <training><generative-model><speech-synthesis>
Title: Adding voices to voice synthesis corpuses
UserDisplayName: Douglas Daseeco
ThreadText: # Adding voices to voice synthesis corpuses
  
  (Asked by: username_0 on 2018-09-16)
  
  If one uses one of the open source implementations of the [WaveNet generative speech synthesis design](https://deepmind.com/blog/wavenet-generative-model-raw-audio/), such as <https://r9y9.github.io/wavenet_vocoder/>, and trains using something like the [CMU's arctic corpus](http://festvox.org/cmu_arctic/), now can one add a voice that sounds younger, older, less professional, or in some other way distinctive. Must the entire training begin from scratch, or is there a more resource and time friendly way?
  
  **Tags:** training, generative-model, speech-synthesis
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-10-31. Score: 1
  
  The original wavenet and the implementation you linked to is globally conditioned on speaker embeddings, which means that the network was given a unique identifier for the person speaking each time it trained on an audio clip. This allows the network to learn to mimic the voice of each person in the training data, but it only learns *their* voices, not arbitrary people's voices.
  
  You might be able to do what you're describing by globally conditioning on different features of the speakers, such as age etc, though this would require a lot a speakers to span those dimensions.
  
  I think it would definitely be possible to use style transfer for this, similar to the work done on tweaking people's age in images [1]. Some similar work has been done for audio in order to change the accent of a voices between accents [2], so I wouldn't be surprised to see more examples of this in the near future.
  
  [1] <https://www.youtube.com/watch?v=2qMw8sOsNg0>
  [2] <https://ai.googleblog.com/2018/03/expressive-speech-synthesis-with.html>
  
  
  ---
  
  