Answers.Body: <p>For ASR this flexibility can be important since audio inputs can vary greatly in clarity and context, for example, even English-speaking natives cannot identify &quot;can&quot; vs &quot;can't&quot;, &quot;map&quot; vs &quot;nap&quot; without more context in many cases, while for other OpenAI models like Chat here each input token has no ambiguity or loss of information through their communication channel, reducing the need for automatic temperature adjustments as ASR in principle.</p>
  <p>Therefore since the Chat API doesn't explicitly mention any automatic temperature adjustments feature, from above argument I recommend we should not believe Chat has this feature by default.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-11-10T03:58:55.557Z
Answers.LastActivityDate: 2024-11-10T03:58:55.557Z
Answers.UserDisplayName: cinch
Body: <p>I read on <a href="https://platform.openai.com/docs/api-reference/audio/createTranscription#audio-createtranscription-temperature" rel="nofollow noreferrer">https://platform.openai.com/docs/api-reference/audio/createTranscription#audio-createtranscription-temperature</a> (<a href="https://web.archive.org/web/20241109194048/https://platform.openai.com/docs/api-reference/audio/createTranscription#audio-createtranscription-temperature" rel="nofollow noreferrer">mirror</a>):</p>
  <blockquote>
  <p>temperature. number. Optional. Defaults to 0. The sampling temperature, between 0 and 1. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. If set to 0, the model will use <a href="https://en.wikipedia.org/wiki/Log_probability" rel="nofollow noreferrer">log probability</a> to automatically increase the temperature until certain thresholds are hit.</p>
  </blockquote>
  <p>and on <a href="https://platform.openai.com/docs/api-reference/chat/create#chat-create-temperature" rel="nofollow noreferrer">https://platform.openai.com/docs/api-reference/chat/create#chat-create-temperature</a> :</p>
  <blockquote>
  <p>temperature. number or null. Optional. Defaults to 1. What sampling temperature to use, between 0 and 2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. We generally recommend altering this or <code>top_p</code> but not both.</p>
  </blockquote>
  <p>Does the non-ASR API (ASR=Automatic Speech Recognition) also &quot;use <a href="https://en.wikipedia.org/wiki/Log_probability" rel="nofollow noreferrer">log probability</a> to automatically increase the temperature until certain thresholds are hit&quot; when temperature=0, is it that just one OpenAI ASR? Or is there a reason why that <a href="https://ai.stackexchange.com/a/47191/4">technique</a> would be specific to ASR?</p>
  
Tags: <large-language-models><open-ai>
Title: Does the non-ASR OpenAI API also "use log probability to automatically increase the temperature until certain thresholds are hit" when temperature=0?
UserDisplayName: Franck Dernoncourt
ThreadText: # Does the non-ASR OpenAI API also "use log probability to automatically increase the temperature until certain thresholds are hit" when temperature=0?
  
  (Asked by: username_0 on 2024-11-10)
  
  I read on <https://platform.openai.com/docs/api-reference/audio/createTranscription#audio-createtranscription-temperature> ([mirror](https://web.archive.org/web/20241109194048/https://platform.openai.com/docs/api-reference/audio/createTranscription#audio-createtranscription-temperature)):
  
  > temperature. number. Optional. Defaults to 0. The sampling temperature, between 0 and 1. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. If set to 0, the model will use [log probability](https://en.wikipedia.org/wiki/Log_probability) to automatically increase the temperature until certain thresholds are hit.
  
  and on <https://platform.openai.com/docs/api-reference/chat/create#chat-create-temperature> :
  
  > temperature. number or null. Optional. Defaults to 1. What sampling temperature to use, between 0 and 2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. We generally recommend altering this or `top_p` but not both.
  
  Does the non-ASR API (ASR=Automatic Speech Recognition) also "use [log probability](https://en.wikipedia.org/wiki/Log_probability) to automatically increase the temperature until certain thresholds are hit" when temperature=0, is it that just one OpenAI ASR? Or is there a reason why that [technique](https://ai.stackexchange.com/a/47191/4) would be specific to ASR?
  
  **Tags:** large-language-models, open-ai
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-11-10. Score: 2
  
  For ASR this flexibility can be important since audio inputs can vary greatly in clarity and context, for example, even English-speaking natives cannot identify "can" vs "can't", "map" vs "nap" without more context in many cases, while for other OpenAI models like Chat here each input token has no ambiguity or loss of information through their communication channel, reducing the need for automatic temperature adjustments as ASR in principle.
  
  Therefore since the Chat API doesn't explicitly mention any automatic temperature adjustments feature, from above argument I recommend we should not believe Chat has this feature by default.
  
  
  ---
  
  