Answers.Body: <p>When temperature is set to <span class="math-container">$0$</span> the ASR model first selects the most likely transcription based on the logits in the most deterministic way compared to all other valid temperatures. However,  when the model encounters ambiguities during transcription (i.e., the logits don't have a clear peak in probabilities thus multiple options are likely such as &quot;can&quot; vs &quot;can't&quot;), temperature is slightly increased to let the model sample from a broader set of candidate outcomes until it can resolve such ambiguity with its other internal complex phoneme/language models and ranking/filtering/feedback processes.</p>
  <p>Finally since exponential is involved with both logits and temperature to get probability distribution via softmax, using log-probability helps numerical stability to avoid overflow/underflow issues along with some other tricks like <a href="https://en.wikipedia.org/wiki/LogSumExp#log-sum-exp_trick_for_log-domain_calculations" rel="nofollow noreferrer">LogSumExp</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-11-10T01:12:36.257Z
Answers.Text: Thanks! Follow-up: [Does the non-ASR OpenAI API also "use log probability to automatically increase the temperature until certain thresholds are hit" when temperature = 0?](https://ai.stackexchange.com/q/47192/4)
Answers.UserDisplayName: Franck Dernoncourt
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-11-09T21:43:34.057Z
Answers.LastActivityDate: 2024-11-09T21:43:34.057Z
Answers.UserDisplayName: cinch
Body: <p>I read on <a href="https://platform.openai.com/docs/api-reference/audio/createTranscription#audio-createtranscription-temperature" rel="nofollow noreferrer">https://platform.openai.com/docs/api-reference/audio/createTranscription#audio-createtranscription-temperature</a> (<a href="https://web.archive.org/web/20241109194048/https://platform.openai.com/docs/api-reference/audio/createTranscription#audio-createtranscription-temperature" rel="nofollow noreferrer">mirror</a>):</p>
  <blockquote>
  <p>temperature. number. Optional. Defaults to 0. The sampling temperature, between 0 and 1. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. If set to 0, the model will use <a href="https://en.wikipedia.org/wiki/Log_probability" rel="nofollow noreferrer">log probability</a> to automatically increase the temperature until certain thresholds are hit.</p>
  </blockquote>
  <p>What does &quot;use <a href="https://en.wikipedia.org/wiki/Log_probability" rel="nofollow noreferrer">log probability</a> to automatically increase the temperature until certain thresholds are hit&quot; mean when using OpenAI ASR with temperature=0?</p>
  
Tags: <open-ai><speech-recognition><decoder>
Title: What does "use log probability to automatically increase the temperature until certain thresholds are hit" mean with OpenAI ASR with temperature=0
UserDisplayName: Franck Dernoncourt
ThreadText: # What does "use log probability to automatically increase the temperature until certain thresholds are hit" mean with OpenAI ASR with temperature=0
  
  (Asked by: username_0 on 2024-11-09)
  
  I read on <https://platform.openai.com/docs/api-reference/audio/createTranscription#audio-createtranscription-temperature> ([mirror](https://web.archive.org/web/20241109194048/https://platform.openai.com/docs/api-reference/audio/createTranscription#audio-createtranscription-temperature)):
  
  > temperature. number. Optional. Defaults to 0. The sampling temperature, between 0 and 1. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. If set to 0, the model will use [log probability](https://en.wikipedia.org/wiki/Log_probability) to automatically increase the temperature until certain thresholds are hit.
  
  What does "use [log probability](https://en.wikipedia.org/wiki/Log_probability) to automatically increase the temperature until certain thresholds are hit" mean when using OpenAI ASR with temperature=0?
  
  **Tags:** open-ai, speech-recognition, decoder
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-11-09. Score: 2
  
  When temperature is set to $0$ the ASR model first selects the most likely transcription based on the logits in the most deterministic way compared to all other valid temperatures. However, when the model encounters ambiguities during transcription (i.e., the logits don't have a clear peak in probabilities thus multiple options are likely such as "can" vs "can't"), temperature is slightly increased to let the model sample from a broader set of candidate outcomes until it can resolve such ambiguity with its other internal complex phoneme/language models and ranking/filtering/feedback processes.
  
  Finally since exponential is involved with both logits and temperature to get probability distribution via softmax, using log-probability helps numerical stability to avoid overflow/underflow issues along with some other tricks like [LogSumExp](https://en.wikipedia.org/wiki/LogSumExp#log-sum-exp_trick_for_log-domain_calculations).
  
  > **Comments:**
  > **username_0**: Thanks! Follow-up: [Does the non-ASR OpenAI API also "use log probability to automatically increase the temperature until certain thresholds are hit" when temperature = 0?](https://ai.stackexchange.com/q/47192/4)  
  
  ---
  
  