Answers.Body: <p>I would just skip the sigmoid function step.</p>
  
  <p>Consider these two scenarios with three choices with the associated values:
  0.1,10,100 or 1,100,1000
  Given that these scenarios are equivalent in their relativ values, your probabilities should be assigned in the same way. But if you throw in a sigmoid function, the difference between 10 and 100 will be bigger than between 100 and 1000. </p>
  
  <p>Just normalise the values, so that they sum to 1. In that case the choice with a value ten times higher than the value of another choice will be picked ten times as often. Makes sense to me. </p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-03-13T16:32:33.657Z
Answers.LastActivityDate: 2017-03-13T16:32:33.657Z
Answers.UserDisplayName: BlindKungFuMaster
Body: <p>I'm using a NN created with CNTK's SimpleNetworkBuilder to make choices (specifically in board games). I specified ReLU as the layer type, so outputs can be arbitrary numbers.</p>
  
  <p>When evaluating a custom set of features, getting the "choice" of the function/model is simple: Look for the output signal with the highest value. However, there are times when I wish to introduce some randomness and assign probabilities to each output signal, then select the choice based on each output's probability.</p>
  
  <p>Currently, what I'm doing is manually normalizing all the output using a sigmoid function specified here: <a href="https://en.wikipedia.org/wiki/Logistic_function" rel="nofollow noreferrer">https://en.wikipedia.org/wiki/Logistic_function</a>
  Then, I multiply them all by a scalar such that the sum total of all outputs is 1.</p>
  
  <p>At this point, I pick a random number 0..1, and see where along the map it falls; that is my selected choice.</p>
  
  <p>What I'd like to know is, is there a better way?</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-03-14T10:05:55.003Z
Comments.Text: "softmax" function
Comments.UserDisplayName: Kh40tiK
Tags: <neural-networks>
Title: Assigning probability to output of a ReLU network
UserDisplayName: Benjamin Chambers
ThreadText: # Assigning probability to output of a ReLU network
  
  (Asked by: username_0 on 2017-03-12)
  
  I'm using a NN created with CNTK's SimpleNetworkBuilder to make choices (specifically in board games). I specified ReLU as the layer type, so outputs can be arbitrary numbers.
  
  When evaluating a custom set of features, getting the "choice" of the function/model is simple: Look for the output signal with the highest value. However, there are times when I wish to introduce some randomness and assign probabilities to each output signal, then select the choice based on each output's probability.
  
  Currently, what I'm doing is manually normalizing all the output using a sigmoid function specified here: <https://en.wikipedia.org/wiki/Logistic_function>
  Then, I multiply them all by a scalar such that the sum total of all outputs is 1.
  
  At this point, I pick a random number 0..1, and see where along the map it falls; that is my selected choice.
  
  What I'd like to know is, is there a better way?
  
  **Tags:** neural-networks
  
  > **Comments:**
  > **username_1**: "softmax" function  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2017-03-13. Score: 1
  
  I would just skip the sigmoid function step.
  
  Consider these two scenarios with three choices with the associated values:
  0.1,10,100 or 1,100,1000
  Given that these scenarios are equivalent in their relativ values, your probabilities should be assigned in the same way. But if you throw in a sigmoid function, the difference between 10 and 100 will be bigger than between 100 and 1000.
  
  Just normalise the values, so that they sum to 1. In that case the choice with a value ten times higher than the value of another choice will be picked ten times as often. Makes sense to me.
  
  
  ---
  
  