Answers.Body: <p>Yes, you can see a comment from discussion here:</p>
  <p><a href="https://github.com/openai/whisper/discussions/81" rel="nofollow noreferrer">https://github.com/openai/whisper/discussions/81</a></p>
  <blockquote>
  <p>This happens when the model is unsure about the output (according to
  the compression_ratio_threshold and logprob_threshold settings). The
  most common failure mode is that it falls into a repeat loop, where it
  likely triggers the compression_ratio_threshold. The default setting
  tries temperatures 0, 0.2, 0.4, 0.6, 0.8, 1.0 until it gives up, at
  which it is less likely to be in a repeat loop but is also less likely
  to be correct.</p>
  </blockquote>
  <p>I've seen also non deterministic results on audio that was clear. Even more than that it generates non related outputs. Using the whisper.ai API you can not play with the parameters but if you are using the open source models you can fine tune it a little bit to perform better.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-16T13:35:10.743Z
Answers.Text: I realised that if you use two threads it splits the sound into two pieces and lets one thread start from the beginning while the other thread starts from the middle. What happens with the second thread is that, that is different from using only one thread, it is missing linguistic context. That is, if you only use one thread, when you reach the middle of the sound file it has some understanding of the subject of the speech it transcribes which it uses when it interprets the speech, which (hopefully) improves the result. I think that is what happened in my case.
Answers.UserDisplayName: d-b
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-15T12:47:26.447Z
Answers.LastActivityDate: 2023-03-16T13:57:47.857Z
Answers.LastEditDate: 2023-03-16T13:57:47.857Z
Answers.UserDisplayName: MikeL
Body: <p>I am experimenting with <a href="https://github.com/openai/whisper" rel="nofollow noreferrer">Whisper AI</a> sound-to-text (actually I am using the <a href="https://github.com/ggerganov/whisper.cpp/" rel="nofollow noreferrer">whisper.cpp</a> clone right now) and am trying to optimize the performance but I have discovered something odd - although I use exactly the same setting every time except for the number of threads, the result isn't exactly the same. In other words, the result varies depending on the number of threads.</p>
  <p>Is this expected? If so, why does it happen?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-02-11T10:26:16.837Z
Comments.Text: Can you please provide a link to this Whisper AI?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-02-11T12:07:56.263Z
Comments.Text: @nbro Added links.
Comments.UserDisplayName: d-b
Tags: <randomness>
Title: Is whisper.ai non-deterministic?
UserDisplayName: d-b
ThreadText: # Is whisper.ai non-deterministic?
  
  (Asked by: username_0 on 2023-02-10)
  
  I am experimenting with [Whisper AI](https://github.com/openai/whisper) sound-to-text (actually I am using the [whisper.cpp](https://github.com/ggerganov/whisper.cpp/) clone right now) and am trying to optimize the performance but I have discovered something odd - although I use exactly the same setting every time except for the number of threads, the result isn't exactly the same. In other words, the result varies depending on the number of threads.
  
  Is this expected? If so, why does it happen?
  
  **Tags:** randomness
  
  > **Comments:**
  > **username_1**: Can you please provide a link to this Whisper AI?  
  > **username_0**: @username_1 Added links.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2023-03-15. Score: 2
  
  Yes, you can see a comment from discussion here:
  
  <https://github.com/openai/whisper/discussions/81>
  
  > This happens when the model is unsure about the output (according to
  > the compression\_ratio\_threshold and logprob\_threshold settings). The
  > most common failure mode is that it falls into a repeat loop, where it
  > likely triggers the compression\_ratio\_threshold. The default setting
  > tries temperatures 0, 0.2, 0.4, 0.6, 0.8, 1.0 until it gives up, at
  > which it is less likely to be in a repeat loop but is also less likely
  > to be correct.
  
  I've seen also non deterministic results on audio that was clear. Even more than that it generates non related outputs. Using the whisper.ai API you can not play with the parameters but if you are using the open source models you can fine tune it a little bit to perform better.
  
  > **Comments:**
  > **username_0**: I realised that if you use two threads it splits the sound into two pieces and lets one thread start from the beginning while the other thread starts from the middle. What happens with the second thread is that, that is different from using only one thread, it is missing linguistic context. That is, if you only use one thread, when you reach the middle of the sound file it has some understanding of the subject of the speech it transcribes which it uses when it interprets the speech, which (hopefully) improves the result. I think that is what happened in my case.  
  
  ---
  
  