Answers.Body: <p>Yes it can. However, other differences between training and test data with audio could have greater effect:</p>
  
  <ul>
  <li><p>Identity of the speaker (including effects from gender, age, physical build, local accent, amongst others)</p></li>
  <li><p>Acoustics of the recording environment (including proximity to the microphone, size of space, presence of hard surfaces, background noise)</p></li>
  </ul>
  
  <p>If any of these may vary from your training data, then it becomes harder to predict your generalised accuracy during training and early model selection.</p>
  
  <p>One possibility is to ensure your cross-validation set (which you absolutely should have) also separates data out by things that will vary from training to test. So instead of random train/cv split, you split by data that is key for generalisation. This is sometimes called a <em>stratified</em> train/test split.</p>
  
  <p>If your only concern is variation in microphone, then split your train/cv sets by microphone type. You will get a better assessment early on in the model selection process how well the training is generalising, and can focus your search on models that do well despite this expected difference.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-02-02T19:29:42.600Z
Answers.LastActivityDate: 2018-02-02T19:39:21.110Z
Answers.LastEditDate: 2018-02-02T19:39:21.110Z
Answers.UserDisplayName: Neil Slater
Answers.Body: <p>The most usual differences in signal records caused by different microphones will have small if not null impact in the recognition accuracy, in particular if we are talking about changes one mic by another of same model and manufactor:</p>
  
  <ul>
  <li>Differences in bandwidth: voice is in a very common (central) bandwidth, it is not expected these differences impacts, even for low quality microphones.</li>
  <li>Microphone distortions: same as previous, they will not impact because they are smaller than, by example, a change in the speaker.</li>
  </ul>
  
  <p>However, if we talk about a general recognition system to be used with very different types of mics, there are some microphone issues that can cause your system completely fail:</p>
  
  <ul>
  <li>mic sensitivity: small sensitivity differences will have no effect because they are solved in the same way than differences in speaker volume/intonation. However, if the microphone is not enough sensible the S/N can be below the minimum need, in particular when speaker increase the distance to the mic.</li>
  <li>lack of beam-forming: if your system is prepared to use an array of microphones to filter noise and/or secondary sources, usage of a normal phone will decrease accuracy.</li>
  <li>changes in sample ratio and/or sample bits: if the microphone and its A/D has a low sampling speed or size (i.e. Bluetooth mics, phone lines, ...), the accuracy can fail.</li>
  </ul>
  
  <p>By example, for IOT applications, the first two of this list are the more challenging ones.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-02-10T18:49:32.977Z
Answers.Text: @DuttaA, OP ask for practical experience: after years working with IOT microphones and voice recognizers, I think I known a few the most critical points about them. However, answer updated with some clarifications.
Answers.UserDisplayName: pasaba por aqui
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-02-10T16:08:14.443Z
Answers.LastActivityDate: 2018-02-10T18:48:34.000Z
Answers.LastEditDate: 2018-02-10T18:48:34.000Z
Answers.UserDisplayName: pasaba por aqui
Body: <p>If I train a speech recognition model using data collected from N different microphones, but deploy it on an unseen (test) microphone - does it impact the accuracy of the model? </p>
  
  <p>While I understand that theoretically an accuracy loss is likely, does anyone have any practical experience with this problem? </p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-02-02T16:05:02.113Z
Comments.Text: Yes it should. I would guess that if you normalize the entire data set then it should alleviate things somewhat but I have no evidence for that belief.
Comments.UserDisplayName: k.c. sayz 'k.c sayz'
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-02-02T19:35:21.497Z
Comments.Text: Welcome to AI!  Great practical question.  I don't have practical experience with this re: AI, but from years of recording experience, I can tell you that microphone quality can greatly affect the quality of the input.  As an analogy, at my office we have a PDF-to-Word function on the copier for scanned documents.  Comparing the output to Adobe's also poor conversion capability, I noticed that the scanning introduced significantly more errors to the same document, no doubt a function of the extra noise and loss of fidelity in the scan.
Comments.UserDisplayName: DukeZhou
Tags: <deep-learning><voice-recognition>
Title: Can variations in microphones used in training set and test set impact the accuracy of speech recognition models?
UserDisplayName: baiduguy1
ThreadText: # Can variations in microphones used in training set and test set impact the accuracy of speech recognition models?
  
  (Asked by: username_0 on 2018-02-02)
  
  If I train a speech recognition model using data collected from N different microphones, but deploy it on an unseen (test) microphone - does it impact the accuracy of the model?
  
  While I understand that theoretically an accuracy loss is likely, does anyone have any practical experience with this problem?
  
  **Tags:** deep-learning, voice-recognition
  > **Comments:**
  > **username_1**: Yes it should. I would guess that if you normalize the entire data set then it should alleviate things somewhat but I have no evidence for that belief.  
  > **username_2**: Welcome to AI!  Great practical question.  I don't have practical experience with this re: AI, but from years of recording experience, I can tell you that microphone quality can greatly affect the quality of the input.  As an analogy, at my office we have a PDF-to-Word function on the copier for scanned documents.  Comparing the output to Adobe's also poor conversion capability, I noticed that the scanning introduced significantly more errors to the same document, no doubt a function of the extra noise and loss of fidelity in the scan.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_3 on 2018-02-02. Score: 3
  
  Yes it can. However, other differences between training and test data with audio could have greater effect:
  
  * Identity of the speaker (including effects from gender, age, physical build, local accent, amongst others)
  * Acoustics of the recording environment (including proximity to the microphone, size of space, presence of hard surfaces, background noise)
  
  If any of these may vary from your training data, then it becomes harder to predict your generalised accuracy during training and early model selection.
  
  One possibility is to ensure your cross-validation set (which you absolutely should have) also separates data out by things that will vary from training to test. So instead of random train/cv split, you split by data that is key for generalisation. This is sometimes called a *stratified* train/test split.
  
  If your only concern is variation in microphone, then split your train/cv sets by microphone type. You will get a better assessment early on in the model selection process how well the training is generalising, and can focus your search on models that do well despite this expected difference.
  
  
  ---
  
  ### Answer by username_4 on 2018-02-10. Score: 1
  
  The most usual differences in signal records caused by different microphones will have small if not null impact in the recognition accuracy, in particular if we are talking about changes one mic by another of same model and manufactor:
  
  * Differences in bandwidth: voice is in a very common (central) bandwidth, it is not expected these differences impacts, even for low quality microphones.
  * Microphone distortions: same as previous, they will not impact because they are smaller than, by example, a change in the speaker.
  
  However, if we talk about a general recognition system to be used with very different types of mics, there are some microphone issues that can cause your system completely fail:
  
  * mic sensitivity: small sensitivity differences will have no effect because they are solved in the same way than differences in speaker volume/intonation. However, if the microphone is not enough sensible the S/N can be below the minimum need, in particular when speaker increase the distance to the mic.
  * lack of beam-forming: if your system is prepared to use an array of microphones to filter noise and/or secondary sources, usage of a normal phone will decrease accuracy.
  * changes in sample ratio and/or sample bits: if the microphone and its A/D has a low sampling speed or size (i.e. Bluetooth mics, phone lines, ...), the accuracy can fail.
  
  By example, for IOT applications, the first two of this list are the more challenging ones.
  
  > **Comments:**
  > **username_4**: @DuttaA, OP ask for practical experience: after years working with IOT microphones and voice recognizers, I think I known a few the most critical points about them. However, answer updated with some clarifications.  
  
  ---
  
  