Answers.Body: <p>Speaker identification is quite widely researched domain. Modern approach would be to map speaker information to i-vector, a real-valued vector of 200-400 components that characterizes speaker fully. i-vectors allow very precise speaker identification and verification.</p>
  
  <p>For more information you can check i-vector <a href="http://www1.icsi.berkeley.edu/Speech/presentations/AFRL_ICSI_visit2_JFA_tutorial_icsitalk.pdf" rel="nofollow noreferrer">tutorial</a></p>
  
  <p>Also you can check state of the art in the results of <a href="https://ivectorchallenge.nist.gov" rel="nofollow noreferrer">NIST i-vector challenge</a></p>
  
  <p>For implementation, you can check the following <a href="https://github.com/kaldi-asr/kaldi/tree/master/egs/sre10/v2" rel="nofollow noreferrer">speaker recognition experiment</a> from Kaldi.</p>
  
  <p>For best accuracy i-vectors are extracted with DNN UBMs, watch out that GMM UBMs are less accurate.</p>
  
  <p>For more in-depth information about speaker recognition methods and algorithms check this <a href="http://rads.stackoverflow.com/amzn/click/0387775919" rel="nofollow noreferrer">textbook</a>.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-11-04T21:28:11.500Z
Answers.LastActivityDate: 2016-11-04T21:28:11.500Z
Answers.UserDisplayName: Nikolay Shmyrev
Answers.Body: <p>Deepmind recently created <a href="https://deepmind.com/blog/wavenet-generative-model-raw-audio/" rel="nofollow noreferrer">a voice synthesiser</a> along those lines. 
  It seems to be incredibly slow, but it might be possible to create a dumped down version of it.</p>
  
  <p>Apparently the task is called parametric TTS (text to speech). <a href="http://mlsp.cs.cmu.edu/courses/fall2012/lectures/spss_specom.pdf" rel="nofollow noreferrer">This overview</a> might give you some leads.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-11-04T15:39:32.330Z
Answers.LastActivityDate: 2016-11-04T15:39:32.330Z
Answers.UserDisplayName: BlindKungFuMaster
Answers.Body: <p>Yes, there is. A quick search found this:
  <a href="https://www.researchgate.net/publication/221536362_Multimodal_Speaker_Identification_Based_on_Text_and_Speech" rel="nofollow noreferrer">Multimodal Speaker Identification Based on Text and Speech</a> (2008).</p>
  <p>In the abstract, they write</p>
  <blockquote>
  <p>This paper proposes a novel method for speaker identification based on both speech utterances and their transcribed text. The transcribed text of each speaker’s utterance is processed by the probabilistic latent semantic indexing (PLSI) that offers a powerful means to model each speaker’s vocabulary employing a number of hidden topics, which are closely related to his/her identity, function, or expertise. Melfrequency cepstral coefficients (MFCCs) are extracted from each speech frame and their dynamic range is quantized to a number of predefined bins in order to compute MFCC local histograms for each speech utterance, which is time-aligned with the transcribed text. Two identity scores are independently computed by the PLSI applied to the text and the nearest neighbor classifier applied to the local MFCC histograms. It is demonstrated that a convex combination of the two scores is more accurate than the individual scores on speaker identification experiments conducted on broadcast news of the RT-03 MDE Training Data Text and Annotations corpus distributed by the Linguistic Data Consortium.</p>
  </blockquote>
  <p>Under figure 2, they write</p>
  <blockquote>
  <p>Identification rate versus Probe ID when 44 speakers are employed. Average identification rates for (a) PLSI: 69%; (b) MFCCs: 66%; (c) Both: 67%.</p>
  </blockquote>
  <p>In section 4, they write</p>
  <blockquote>
  <p>To demonstrate the proposed multimodal speaker identification algorithm, experiments are conducted on broadcast news (BN) collected within the DARPA Efficient, Affordable, Reusable Speech-to-Text (EARS) Program in Metadata Extraction (MDE).</p>
  </blockquote>
  <p>If you need more papers related, you could use a tool like <a href="https://the.iris.ai/" rel="nofollow noreferrer">https://the.iris.ai/</a> to find related papers.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2016-11-04T15:39:39.847Z
Answers.LastActivityDate: 2022-04-11T22:18:43.020Z
Answers.LastEditDate: 2022-04-11T22:18:43.020Z
Answers.UserDisplayName: Sergi C
Body: <p>Has there been research done regarding processing speech then building a "speaker profile" based off the processed speech? Things like matching the voice with a speaker profile and matching speech patterns and wordage for the speaker profile would be examples of building the profile. Basically, building a model of an individual based solely off speech. Any examples of this being implemented would be greatly appreciated.</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2016-11-04T22:03:31.200Z
Comments.Text: NNs are pretty powerful for this problem. Check this: https://deepmind.com/blog/wavenet-generative-model-raw-audio/ and the implementation here: https://github.com/ibab/tensorflow-wavenet
Comments.UserDisplayName: Bobi Rakova
Tags: <natural-language-processing><reference-request><speech-recognition><voice-recognition>
Title: Has there been research done regarding processing speech then building a "speaker profile" based off the processed speech?
UserDisplayName: Tory
ThreadText: # Has there been research done regarding processing speech then building a "speaker profile" based off the processed speech?
  
  (Asked by: username_0 on 2016-11-04)
  
  Has there been research done regarding processing speech then building a "speaker profile" based off the processed speech? Things like matching the voice with a speaker profile and matching speech patterns and wordage for the speaker profile would be examples of building the profile. Basically, building a model of an individual based solely off speech. Any examples of this being implemented would be greatly appreciated.
  
  **Tags:** natural-language-processing, reference-request, speech-recognition, voice-recognition
  
  > **Comments:**
  > **username_1**: NNs are pretty powerful for this problem. Check this: https://deepmind.com/blog/wavenet-generative-model-raw-audio/ and the implementation here: https://github.com/ibab/tensorflow-wavenet  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_4 on 2016-11-04. Score: 2 | Accepted answer
  
  Yes, there is. A quick search found this:
  [Multimodal Speaker Identification Based on Text and Speech](https://www.researchgate.net/publication/221536362_Multimodal_Speaker_Identification_Based_on_Text_and_Speech) (2008).
  
  In the abstract, they write
  
  > This paper proposes a novel method for speaker identification based on both speech utterances and their transcribed text. The transcribed text of each speaker’s utterance is processed by the probabilistic latent semantic indexing (PLSI) that offers a powerful means to model each speaker’s vocabulary employing a number of hidden topics, which are closely related to his/her identity, function, or expertise. Melfrequency cepstral coefficients (MFCCs) are extracted from each speech frame and their dynamic range is quantized to a number of predefined bins in order to compute MFCC local histograms for each speech utterance, which is time-aligned with the transcribed text. Two identity scores are independently computed by the PLSI applied to the text and the nearest neighbor classifier applied to the local MFCC histograms. It is demonstrated that a convex combination of the two scores is more accurate than the individual scores on speaker identification experiments conducted on broadcast news of the RT-03 MDE Training Data Text and Annotations corpus distributed by the Linguistic Data Consortium.
  
  Under figure 2, they write
  
  > Identification rate versus Probe ID when 44 speakers are employed. Average identification rates for (a) PLSI: 69%; (b) MFCCs: 66%; (c) Both: 67%.
  
  In section 4, they write
  
  > To demonstrate the proposed multimodal speaker identification algorithm, experiments are conducted on broadcast news (BN) collected within the DARPA Efficient, Affordable, Reusable Speech-to-Text (EARS) Program in Metadata Extraction (MDE).
  
  If you need more papers related, you could use a tool like <https://the.iris.ai/> to find related papers.
  
  
  ---
  
  ### Answer by username_2 on 2016-11-04. Score: 1
  
  Speaker identification is quite widely researched domain. Modern approach would be to map speaker information to i-vector, a real-valued vector of 200-400 components that characterizes speaker fully. i-vectors allow very precise speaker identification and verification.
  
  For more information you can check i-vector [tutorial](http://www1.icsi.berkeley.edu/Speech/presentations/AFRL_ICSI_visit2_JFA_tutorial_icsitalk.pdf)
  
  Also you can check state of the art in the results of [NIST i-vector challenge](https://ivectorchallenge.nist.gov)
  
  For implementation, you can check the following [speaker recognition experiment](https://github.com/kaldi-asr/kaldi/tree/master/egs/sre10/v2) from Kaldi.
  
  For best accuracy i-vectors are extracted with DNN UBMs, watch out that GMM UBMs are less accurate.
  
  For more in-depth information about speaker recognition methods and algorithms check this [textbook](http://rads.stackoverflow.com/amzn/click/0387775919).
  
  
  ---
  
  ### Answer by username_3 on 2016-11-04. Score: 0
  
  Deepmind recently created [a voice synthesiser](https://deepmind.com/blog/wavenet-generative-model-raw-audio/) along those lines.
  It seems to be incredibly slow, but it might be possible to create a dumped down version of it.
  
  Apparently the task is called parametric TTS (text to speech). [This overview](http://mlsp.cs.cmu.edu/courses/fall2012/lectures/spss_specom.pdf) might give you some leads.
  
  
  ---
  
  