Answers.Body: <p>If you have a finite set of possibilities (such as 0/1 for binary), the entropy is maximized, given that the probability of observing one or the other is 0.5, and nothing can increase it further</p>
  <p>If you have a distribution that is close to the uniform, your entropy is high, but it's not maximized</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-26T08:37:52.593Z
Answers.LastActivityDate: 2024-01-26T08:37:52.593Z
Answers.UserDisplayName: Alberto
Body: <p>If the entropy of information reaches its maximum when all values of X are equally probable, indicating maximum uncertainty, do probabilities that don't vary much from the average contribute to greater entropy?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-01-26T11:03:50.107Z
Comments.Text: Can you please explain how this is related to AI? I know that this can be related to AI but it might not. So, please, provide more info about why you're asking this question.
Comments.UserDisplayName: nbro
Tags: <information-theory><entropy>
Title: Do probabilities that don't vary much from the average contribute to greater entropy?
UserDisplayName: MAIKAO
ThreadText: # Do probabilities that don't vary much from the average contribute to greater entropy?
  
  (Asked by: username_0 on 2024-01-26)
  
  If the entropy of information reaches its maximum when all values of X are equally probable, indicating maximum uncertainty, do probabilities that don't vary much from the average contribute to greater entropy?
  
  **Tags:** information-theory, entropy
  
  > **Comments:**
  > **username_1**: Can you please explain how this is related to AI? I know that this can be related to AI but it might not. So, please, provide more info about why you're asking this question.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2024-01-26. Score: 0
  
  If you have a finite set of possibilities (such as 0/1 for binary), the entropy is maximized, given that the probability of observing one or the other is 0.5, and nothing can increase it further
  
  If you have a distribution that is close to the uniform, your entropy is high, but it's not maximized
  
  
  ---
  
  