Answers.Body: <p>LLMs generate probability density over the vocabulary of tokens that can be generated, i.e. for a given input text, they output a set of probabilities corresponding to each token in the vocabulary. The higher probability means that the LLM predicts it to be a more reliable next token than the lower probability ones. For example, given the input text, <code>She has a pet</code>, the probability might look like <code>{'dog': 0.4, 'cat': 0.3, 'snake': 0.2, 'crocodile': 0.1}</code> just as an illustration. Now, if you have tried out OpenAI's playground, they offer an interactive slider to play around with the variables, one of which is <code>temperature</code>. The less the temperature, the more greedy the model, i.e. if you set the temperature to 0, the LLM will always choose the token with the highest probability, no matter what. And so, given the temperature is 0, every time you ask the same question, the LLM-generated response will be the same.</p>
  <p>Alternatively, increasing the temperature introduces randomness. Now, the model, instead of selecting the token with the highest probability, will choose one of the options from the top n tokens with the highest probabilities; it becomes stochastic. So, for the same given question, the next token will be randomly selected and therefore the ultimate response will be different.</p>
  <p>BARD and ChatGPT use stochastic selections to make them more creative and also to appeal to the masses. It makes the LLM feel <code>sentient</code>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-07-22T05:33:38.050Z
Answers.Text: Thank you for the answer, I’m tempted to ask how prompts like ”write me a song about rain” are handled but I rather ask is it LLM which answers such questions at all?
Answers.UserDisplayName: harism
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-07-22T05:37:11.380Z
Answers.Text: The question seems unclear. But, the prompt `write me a song about rain` is answered exactly the same way. The LLM outputs a list of probabilities, and the selection criteria select the next token, and this repeats until a stop token is reached. So yes, LLMs answer these questions.
Answers.UserDisplayName: Chinmay
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-07-22T06:12:53.893Z
Answers.Text: Your example question gives interesting answers by the way and I need to go repeat it a few times once got time. Cheers!
Answers.UserDisplayName: harism
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-10-04T13:57:09.293Z
Answers.Text: I set temperature to 1.7 expecting different randomness in responses and I get the same answers with no randomness. Randomness is just not something ChatGPT can do.
Answers.UserDisplayName: AlxVallejo
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-07-22T05:27:04.187Z
Answers.LastActivityDate: 2023-07-22T05:27:04.187Z
Answers.UserDisplayName: Chinmay
Body: <p>I tried Google Bard now recently for the first time after we got into here in Finland too. And to my surprise it gives different answer each time I ask the same question. Is this a feature of LLM or Bard or what takes?</p>
  
Tags: <chatgpt>
Title: Randomness in Google Bard answers
UserDisplayName: harism
ThreadText: # Randomness in Google Bard answers
  
  (Asked by: username_0 on 2023-07-22)
  
  I tried Google Bard now recently for the first time after we got into here in Finland too. And to my surprise it gives different answer each time I ask the same question. Is this a feature of LLM or Bard or what takes?
  
  **Tags:** chatgpt
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-07-22. Score: 4 | Accepted answer
  
  LLMs generate probability density over the vocabulary of tokens that can be generated, i.e. for a given input text, they output a set of probabilities corresponding to each token in the vocabulary. The higher probability means that the LLM predicts it to be a more reliable next token than the lower probability ones. For example, given the input text, `She has a pet`, the probability might look like `{'dog': 0.4, 'cat': 0.3, 'snake': 0.2, 'crocodile': 0.1}` just as an illustration. Now, if you have tried out OpenAI's playground, they offer an interactive slider to play around with the variables, one of which is `temperature`. The less the temperature, the more greedy the model, i.e. if you set the temperature to 0, the LLM will always choose the token with the highest probability, no matter what. And so, given the temperature is 0, every time you ask the same question, the LLM-generated response will be the same.
  
  Alternatively, increasing the temperature introduces randomness. Now, the model, instead of selecting the token with the highest probability, will choose one of the options from the top n tokens with the highest probabilities; it becomes stochastic. So, for the same given question, the next token will be randomly selected and therefore the ultimate response will be different.
  
  BARD and ChatGPT use stochastic selections to make them more creative and also to appeal to the masses. It makes the LLM feel `sentient`.
  
  > **Comments:**
  > **username_0**: Thank you for the answer, I’m tempted to ask how prompts like ”write me a song about rain” are handled but I rather ask is it LLM which answers such questions at all?  
  > **username_1**: The question seems unclear. But, the prompt `write me a song about rain` is answered exactly the same way. The LLM outputs a list of probabilities, and the selection criteria select the next token, and this repeats until a stop token is reached. So yes, LLMs answer these questions.  
  > **username_0**: Your example question gives interesting answers by the way and I need to go repeat it a few times once got time. Cheers!  
  > **username_2**: I set temperature to 1.7 expecting different randomness in responses and I get the same answers with no randomness. Randomness is just not something ChatGPT can do.  
  
  ---
  
  