Answers.Body: <p>Excellent question.</p>
  <p>The simple answer is <strong>no</strong>.  Softmax actually produces uncalibrated probabilities.  That is, they do not really represent the probability of a prediction being correct.</p>
  <p>What usually happens is that softmax probabilities for the predicted class are closer to 100% in all cases, whether the predictions are correct or incorrect, which effectively does not give you any information. This is called overconfidence.</p>
  <p>This means that the probabilities are not useful, and you cannot really use them as reliable confidences to detect when the model is unsure or predicts incorrectly.</p>
  <p>For reference: Guo C, Pleiss G, Sun Y, Weinberger KQ. <a href="http://proceedings.mlr.press/v70/guo17a.html" rel="noreferrer">On calibration of modern neural networks</a>. In International conference on machine learning 2017 Jul 17 (pp. 1321-1330). PMLR.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-16T09:53:51.053Z
Answers.Text: Related question: If I include several copies of each image in my training set, and label some copies to some classes, and other copies to other classes, should I expect the softmax output to be close to the distributions in the training set? For instance, if a particular image has 4 copies in the training set, with 3 copies in class A and 1 copy in class B, and I run the classifier on this image, should I expect the output on that image to be close to 75% for A and 25% for B?
Answers.UserDisplayName: Stef
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-16T18:04:46.157Z
Answers.Text: @Stef This will surely confuse the model but I do not think there is something generalizable about what would happen, it depends on the model, so you should not have an expectation. Label smoothing or mixup training would have the effect you mention.
Answers.UserDisplayName: Dr. Snoopy
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-12-14T18:06:13.163Z
Answers.Text: I'd like to point out that the original post contains 2 questions, in my view, and you're answering the second one correctly. First, we must say that, by the definition of a probability or probability vector, the softmax produces a probability vector and the sigmoid produces a probability (number between 0 and 1). Here, you answer the more relevant (in my view) question: do these mentioned probabilities represent good/true confidence/uncertainty values?
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-14T20:42:31.057Z
Answers.LastActivityDate: 2022-11-14T22:51:08.170Z
Answers.LastEditDate: 2022-11-14T22:51:08.170Z
Answers.UserDisplayName: Dr. Snoopy
Answers.Body: <p><strong>Yes</strong>, but as the other answers said, the raw softmax output has the issue of <strong>overconfidence</strong>. So, you need a <strong>transformation</strong> to make them linear like <strong>logarithmic scale</strong>.</p>
  <p>Logarithms have a unique relationship with infinity. As numbers approach infinity, logarithms grow, but they do so more slowly than the numbers themselves. This slow growth can be philosophically interpreted as a metaphor for how small actions or small steps can still have an infinite impact over time. <strong>The logarithm provides a bridge between the finite (real-world measurements like IEEE 754 representation) and the infinite (theoretical concepts)</strong>, suggesting that even seemingly small, finite changes can have profound, infinite consequences.</p>
  <p>To address the overconfidence issue, let <span class="math-container">$a$</span> be the <strong>maximum</strong> value of probability. Intuitively, putting them inside a logarithmic scale <span class="math-container">$ -\log_2(1 - a)$</span> would reduce the overconfidence issue as it's sensitive to the small difference when the <span class="math-container">$a$</span> close to <span class="math-container">$1$</span>, therefore it can be used for <strong>confidence</strong> measurement.</p>
  <p>Additionally, we can also consider the <strong>minimum</strong> value of probability to represent how assertive the softmax output's statement is as the support, and so that I can consider it as an <strong>uncertainty</strong> measurement.</p>
  <hr />
  <p>I tested it when I trained MNIST Handwritten Digit of 10 classes as usual. Where I use the original input and its 90-degree-rotated (transposed) input. Subjectively speaking, there is no such number that looks like a horizontal line for the class of digit number 1 when it's rotated while the class of digit number 0 is similar like its original when it's rotated.</p>
  <p><a href="https://i.sstatic.net/lGPME7A9.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/lGPME7A9.png" alt="enter image description here" /></a></p>
  <hr />
  <p>Performing a transformation of raw softmax output into a logarithmic scale would produce probabilities of <em>I don't know</em> vs. <em>I know</em>, like this class of digit number 1 below. Notice that there are <strong>gaps of numerical precision</strong>, as the most real number is represented by IEEE 754 in most computers, which has limited precision.</p>
  <p><a href="https://i.sstatic.net/jOKRqFdJ.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/jOKRqFdJ.png" alt="enter image description here" /></a></p>
  <hr />
  <p>Same for the class of digit number 0 below as expected, it is mostly correct to predict its rotated version as it's similar to the original.</p>
  <p><a href="https://i.sstatic.net/ykDh9bf0.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/ykDh9bf0.png" alt="enter image description here" /></a></p>
  <hr />
  <p>Here is the full formula:</p>
  <h3>Inputs:</h3>
  <ul>
  <li>Let <span class="math-container">$\mathbf{p} \in \mathbb{R}^{n_{\text{samples}} \times n_{\text{classes}}}$</span> be the input matrix of predicted probabilities, where:
  <ul>
  <li><span class="math-container">$n_{\text{samples}}$</span> is the number of samples.</li>
  <li><span class="math-container">$n_{\text{classes}}$</span> is the number of classes (i.e., the number of possible predictions per sample).</li>
  <li>Each element <span class="math-container">$p_{i,j} \in [0, 1]$</span> represents the predicted probability for sample <span class="math-container">$i$</span> and class <span class="math-container">$j$</span>, and for each <span class="math-container">$i$</span>, we have <span class="math-container">$\sum_{j=1}^{n_{\text{classes}}} p_{i,j} = 1$</span>.</li>
  </ul>
  </li>
  </ul>
  <hr />
  <h3>Definitions:</h3>
  <ul>
  <li>Let <span class="math-container">$N = n_{\text{classes}}$</span> denote the number of classes.</li>
  <li>Let <span class="math-container">$\text{inf} = -\log_2(\epsilon)$</span> (where <span class="math-container">$\epsilon$</span> is the smallest representable number for the input's data type)</li>
  <li>For each sample <span class="math-container">$i$</span>, define:
  <ul>
  <li>The maximum predicted probability <span class="math-container">$a_i = \max_j (p_{i,j})$</span>.</li>
  <li>The minimum predicted probability <span class="math-container">$b_i = \min_j (p_{i,j})$</span>.</li>
  </ul>
  </li>
  </ul>
  <hr />
  <h3>Confidence:</h3>
  <p>For each sample <span class="math-container">$i$</span>, the confidence is defined as:</p>
  <p><span class="math-container">$$
  c_i = \log_2(N - 1) - \log_2(N) - \log_2(1 - a_i)
  $$</span></p>
  <p>where <span class="math-container">$a_i = \max_j (p_{i,j})$</span> is the maximum predicted probability for sample <span class="math-container">$i$</span>.</p>
  <p><strong>Specifically:</strong>
  The confidence for each sample is normalized as:</p>
  <p><span class="math-container">$$
  c_i' = \frac{c_i}{\inf}
  $$</span></p>
  <p>where <span class="math-container">$\inf$</span> is a large constant, representing the inverse of the smallest representable number of the data type, ensuring that the confidence is mapped to the range <span class="math-container">$[0, 1]$</span>.</p>
  <hr />
  <h3>Uncertainty:</h3>
  <p>The uncertainty for each sample <span class="math-container">$i$</span> is defined as:</p>
  <p><span class="math-container">$$
  u_i = -\log_2(N) - \log_2(b_i) - c_i
  $$</span></p>
  <p>where <span class="math-container">$b_i = \min_j (p_{i,j})$</span> is the minimum predicted probability for sample <span class="math-container">$i$</span>, and <span class="math-container">$c_i$</span> is the confidence for sample <span class="math-container">$i$</span>.</p>
  <p><strong>Specifically:</strong>
  The uncertainty for each sample is normalized as:</p>
  <p><span class="math-container">$$
  u_i' = \frac{u_i}{\inf}
  $$</span></p>
  <p>where <span class="math-container">$\inf$</span> is the same large constant used for normalizing the confidence.</p>
  <p>Then, the uncertainty is transformed as:</p>
  <p><span class="math-container">$$
  u_i'' = \frac{2^{u_i'} - 1}{2^{u_i'} + 1}
  $$</span></p>
  <p>This maps the uncertainty values to the range <span class="math-container">$[0, 1]$</span>.</p>
  <hr />
  <h3>Outputs:</h3>
  <ul>
  <li>The output consists of two arrays:
  <ol>
  <li><strong>Confidence array <span class="math-container">$\mathbf{c}$</span></strong>: A normalized confidence value for each sample, where <span class="math-container">$c_i' \in [0, 1]$</span>.</li>
  <li><strong>Uncertainty array <span class="math-container">$\mathbf{u}$</span></strong>: A normalized uncertainty value for each sample, where <span class="math-container">$u_i'' \in [0, 1]$</span>.</li>
  </ol>
  </li>
  </ul>
  <p>The final output is a matrix <span class="math-container">$\mathbf{output} \in \mathbb{R}^{n_{\text{samples}} \times 2}$</span>, where each row contains the normalized confidence and uncertainty values for each sample.</p>
  <hr />
  <h1>Summary</h1>
  <p>So, you can say that the softmax output can represent the probability, except it represents the probability of <em>I don't know this sample vs. I know this sample</em>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-12-25T01:17:51.837Z
Answers.LastActivityDate: 2024-12-25T06:13:19.923Z
Answers.LastEditDate: 2024-12-25T06:13:19.923Z
Answers.UserDisplayName: Muhammad Ikhwan Perwira
Answers.Body: <p>The answer is both yes, and no.  Or, to put it another way, the answer depends on what exactly you mean by &quot;represent probabilities&quot;, and there is a valid sense in which the answer is yes, and another valid sense in which the answer is no.</p>
  <h1>No, they don't represent the probability</h1>
  <p>No, they do not represent the true probability.</p>
  <p>You can think of a neural network as a function <span class="math-container">$f$</span>.  Let <span class="math-container">$f(y;x)$</span>  denote the softmax output of the neural network corresponding to class <span class="math-container">$y$</span>, on input <span class="math-container">$x$</span>.  Then <span class="math-container">$f(y;x)$</span> will typically not be equal to <span class="math-container">$p(y|x)$</span>, the probability that sample <span class="math-container">$x$</span> is from class <span class="math-container">$y$</span>.  <span class="math-container">$f(y;x)$</span> can be viewed as an estimate of <span class="math-container">$p(y|x)$</span> -- as a best-effort guess at <span class="math-container">$p(y|x)$</span> -- but it can be an arbitrarily bad estimate/guess.  Neural networks routinely make errors on tasks that even humans find clear.  Also, neural networks have systematic biases.  For instance, as the other answer explains, neural networks tend to be biased towards &quot;overconfidence&quot;.</p>
  <p>So you should not assume that the output from the neural network represents the true probability <span class="math-container">$p(y|x)$</span>.  There is some underlying probability.  We might not know how to compute it, but it exists.  Neural networks are an attempt to estimate it, but it is a highly imperfect estimate.</p>
  <h1>Yes, they do represent probabilities</h1>
  <p>While the softmax outputs are not the true probability <span class="math-container">$p(y|x)$</span>, they do represent a probability distribution.  You can think of them as an estimate of <span class="math-container">$p(y|x)$</span>.  For a number of reasons, it is an imperfect and flawed estimate, but it is an estimate nonetheless.  (Even bad or noisy estimates are still estimates.)</p>
  <p>Moreover, the way we train neural networks is designed to try to make them a good estimate -- or as good as possible.  We train a neural network to minimize the expected loss.  The expected loss is defined as</p>
  <p><span class="math-container">$$L = \mathbb{E}_x[H(p(y|x),f(y;x))],$$</span></p>
  <p>where the expectation is with respect to <span class="math-container">$x$</span> chosen according to the data distribution embodied in the training set, and <span class="math-container">$H$</span> is the <a href="https://en.wikipedia.org/wiki/Cross_entropy" rel="noreferrer">cross-entropy</a> of the distribution <span class="math-container">$f(y;x)$</span> relative to the distribution <span class="math-container">$p(y|x)$</span>.  Intuitively, the smaller the training loss is, the closer that <span class="math-container">$f(y;x)$</span> is to <span class="math-container">$p(y|x)$</span>.</p>
  <p>So, neural networks are trained in a way that tries to make its output be as good an approximation to <span class="math-container">$p(y|x)$</span> as is possible, given the limitations of neural networks and given the training data that is available.  As highlighted above, this is highly imperfect.  But <span class="math-container">$f(y;x)$</span> does still represent a probability distribution, that is our attempt to estimate the true probability distribution <span class="math-container">$p(y|x)$</span>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-12-09T18:04:46.987Z
Answers.Text: "they do represent a probability distribution" -> Why? Is it because of the properties of the output values (positive, summing to 1, etc) or because of it's an inherent part of the softmax? If it's the first, then a bad estimator that always outputs `[0, 0, ..., 0, 1]` can also be considered a probability distribution, right?
Answers.UserDisplayName: Kostas Mouratidis
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-12-09T18:20:14.023Z
Answers.Text: @KostasMouratidis, Yes to both.  The properties of softmax ensure that the output values represent a probability distribution, i.e., each number is in the range [0,1] and they sum to 1.  Yes, [0,0,...,0,1] is a probability distribution.  As you say, it might be a terrible estimate of the $p(y|x)$, but it is a probability distribution.
Answers.UserDisplayName: D.W.
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-15T07:15:29.203Z
Answers.LastActivityDate: 2022-11-15T07:15:29.203Z
Answers.UserDisplayName: D.W.
Body: <p><strong>BACKGROUND:</strong>  The <a href="https://en.wikipedia.org/wiki/Softmax_function" rel="noreferrer">softmax function</a> is the most common choice for an activation function for the last dense layer of a multiclass neural network classifier.  The outputs of the softmax function have mathematical properties of probabilities and are--in practice--presumed to be (conditional) probabilities of the classes given the features:</p>
  <ol>
  <li>First, the softmax output for each class is between <span class="math-container">$0$</span> and <span class="math-container">$1$</span>.</li>
  <li>Second, the outputs of all the classes sum to <span class="math-container">$1$</span>.</li>
  </ol>
  <p><strong>PROBLEM:</strong>  However, just because they have mathematical properties of probabilities does not automatically mean that the softmax outputs are in fact probabilities.  In fact, there are other functions that also have these mathematical properties, which are also occasionally used as activation functions.</p>
  <p><strong>QUESTION:</strong>  <em>&quot;Do softmax outputs represent probabilities in the usual sense?&quot;</em>  In other words, do they really reflect chances or likelihoods? (I use likelihood in the colloquial sense here.)</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-11-15T04:21:08.923Z
Comments.Text: How would you define "represent probabilities"?  What does that mean to you?  What is the difference between  "represent probabilities" vs "have the mathematical properties of probabilities"?
Comments.UserDisplayName: D.W.
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-11-15T04:54:56.173Z
Comments.Text: @D.W., by "represent probabilities", I mean "do they really reflect chances or likelihoods?"  For example, if I toss a die lots of times, I expect each of the six faces to show up 1/6 of the time.  Now let's assume we have a classification problem with 3 classes:  A, B, and C..  If we sample lots of examples with a defined feature set, I might get a% are class A, b% are class B, and c% class C, empirically.   Now, if I feed an example with that same feature set to a neural network, will the softmax probabilities be (a%, b%, c%) ?  According to the answer below, the answer is **no**.
Comments.UserDisplayName: Snehal Patel
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-11-15T07:48:59.707Z
Comments.Text: I do not call this overconfidence, this is called overconfidence in the literature, note that my answer is supported by references. The fact that they are not calibrated means they are not really probabilities (do not represent the true likelihood of the data).
Comments.UserDisplayName: Dr. Snoopy
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-11-15T07:53:54.627Z
Comments.Text: @Dr. Snoopy, yes, I see how my comment was not worded properly.  It gave the wrong impression on where the "overconfidence" term was coming from.  I deleted that comment.  My apologies.
Comments.UserDisplayName: Snehal Patel
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-11-15T07:57:12.190Z
Comments.Text: Anyways the answer you now accepted is plain wrong, not all estimates are useful.
Comments.UserDisplayName: Dr. Snoopy
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-11-15T08:03:45.590Z
Comments.Text: Regarding the issue of calibration, @D.W. points out that the issue of whether the softmax probabilities are "real probabilities" is nuanced.   One the one hand, he presents a view that is in agreement with yours, but he also offers second perspective.  In the end, I am able to hold both views in my mind comfortably.  I thank you for taking the time to provide an excellent response and for providing the reference.
Comments.UserDisplayName: Snehal Patel
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-11-15T08:07:19.010Z
Comments.Text: If you feel that @D.W. is wrong, please update your post accordingly.  I am not immune to making mistakes and I am not immune to changing my mind.  I just need a compelling reason to do so.
Comments.UserDisplayName: Snehal Patel
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-11-16T10:59:32.117Z
Comments.Text: I already said it in my answer, the interpretation of probabilistic classifier is to be the probability of a prediction being correct, that is not how we train models using the cross-entropy loss, so what you get are not probabilities of being correct, specially if a classifier always outputs 100% confidence, this is exactly the same you said for probabilities to be proportional to class imbalance, this also does not happen in practice. Theory can hold anything, but practice is the one that matters.
Comments.UserDisplayName: Dr. Snoopy
Tags: <activation-functions><probability><softmax><probability-theory>
Title: Are softmax outputs of classifiers true probabilities?
UserDisplayName: Snehal Patel
ThreadText: # Are softmax outputs of classifiers true probabilities?
  
  (Asked by: username_0 on 2022-11-14)
  
  **BACKGROUND:** The [softmax function](https://en.wikipedia.org/wiki/Softmax_function) is the most common choice for an activation function for the last dense layer of a multiclass neural network classifier. The outputs of the softmax function have mathematical properties of probabilities and are--in practice--presumed to be (conditional) probabilities of the classes given the features:
  
  1. First, the softmax output for each class is between $0$ and $1$.
  2. Second, the outputs of all the classes sum to $1$.
  
  **PROBLEM:** However, just because they have mathematical properties of probabilities does not automatically mean that the softmax outputs are in fact probabilities. In fact, there are other functions that also have these mathematical properties, which are also occasionally used as activation functions.
  
  **QUESTION:** *"Do softmax outputs represent probabilities in the usual sense?"* In other words, do they really reflect chances or likelihoods? (I use likelihood in the colloquial sense here.)
  
  **Tags:** activation-functions, probability, softmax, probability-theory
  
  > **Comments:**
  > **username_1**: How would you define "represent probabilities"?  What does that mean to you?  What is the difference between  "represent probabilities" vs "have the mathematical properties of probabilities"?  
  > **username_0**: @D.W., by "represent probabilities", I mean "do they really reflect chances or likelihoods?"  For example, if I toss a die lots of times, I expect each of the six faces to show up 1/6 of the time.  Now let's assume we have a classification problem with 3 classes:  A, B, and C..  If we sample lots of examples with a defined feature set, I might get a% are class A, b% are class B, and c% class C, empirically.   Now, if I feed an example with that same feature set to a neural network, will the softmax probabilities be (a%, b%, c%) ?  According to the answer below, the answer is **no**.  
  > **username_2**: I do not call this overconfidence, this is called overconfidence in the literature, note that my answer is supported by references. The fact that they are not calibrated means they are not really probabilities (do not represent the true likelihood of the data).  
  > **username_0**: @username_2, yes, I see how my comment was not worded properly.  It gave the wrong impression on where the "overconfidence" term was coming from.  I deleted that comment.  My apologies.  
  > **username_2**: Anyways the answer you now accepted is plain wrong, not all estimates are useful.  
  > **username_0**: Regarding the issue of calibration, @D.W. points out that the issue of whether the softmax probabilities are "real probabilities" is nuanced.   One the one hand, he presents a view that is in agreement with yours, but he also offers second perspective.  In the end, I am able to hold both views in my mind comfortably.  I thank you for taking the time to provide an excellent response and for providing the reference.  
  > **username_0**: If you feel that @D.W. is wrong, please update your post accordingly.  I am not immune to making mistakes and I am not immune to changing my mind.  I just need a compelling reason to do so.  
  > **username_2**: I already said it in my answer, the interpretation of probabilistic classifier is to be the probability of a prediction being correct, that is not how we train models using the cross-entropy loss, so what you get are not probabilities of being correct, specially if a classifier always outputs 100% confidence, this is exactly the same you said for probabilities to be proportional to class imbalance, this also does not happen in practice. Theory can hold anything, but practice is the one that matters.  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_1 on 2022-11-15. Score: 17 | Accepted answer
  
  The answer is both yes, and no. Or, to put it another way, the answer depends on what exactly you mean by "represent probabilities", and there is a valid sense in which the answer is yes, and another valid sense in which the answer is no.
  
  # No, they don't represent the probability
  
  No, they do not represent the true probability.
  
  You can think of a neural network as a function $f$. Let $f(y;x)$ denote the softmax output of the neural network corresponding to class $y$, on input $x$. Then $f(y;x)$ will typically not be equal to $p(y|x)$, the probability that sample $x$ is from class $y$. $f(y;x)$ can be viewed as an estimate of $p(y|x)$ -- as a best-effort guess at $p(y|x)$ -- but it can be an arbitrarily bad estimate/guess. Neural networks routinely make errors on tasks that even humans find clear. Also, neural networks have systematic biases. For instance, as the other answer explains, neural networks tend to be biased towards "overconfidence".
  
  So you should not assume that the output from the neural network represents the true probability $p(y|x)$. There is some underlying probability. We might not know how to compute it, but it exists. Neural networks are an attempt to estimate it, but it is a highly imperfect estimate.
  
  # Yes, they do represent probabilities
  
  While the softmax outputs are not the true probability $p(y|x)$, they do represent a probability distribution. You can think of them as an estimate of $p(y|x)$. For a number of reasons, it is an imperfect and flawed estimate, but it is an estimate nonetheless. (Even bad or noisy estimates are still estimates.)
  
  Moreover, the way we train neural networks is designed to try to make them a good estimate -- or as good as possible. We train a neural network to minimize the expected loss. The expected loss is defined as
  
  $$L = \mathbb{E}\_x[H(p(y|x),f(y;x))],$$
  
  where the expectation is with respect to $x$ chosen according to the data distribution embodied in the training set, and $H$ is the [cross-entropy](https://en.wikipedia.org/wiki/Cross_entropy) of the distribution $f(y;x)$ relative to the distribution $p(y|x)$. Intuitively, the smaller the training loss is, the closer that $f(y;x)$ is to $p(y|x)$.
  
  So, neural networks are trained in a way that tries to make its output be as good an approximation to $p(y|x)$ as is possible, given the limitations of neural networks and given the training data that is available. As highlighted above, this is highly imperfect. But $f(y;x)$ does still represent a probability distribution, that is our attempt to estimate the true probability distribution $p(y|x)$.
  
  > **Comments:**
  > **username_6**: "they do represent a probability distribution" -> Why? Is it because of the properties of the output values (positive, summing to 1, etc) or because of it's an inherent part of the softmax? If it's the first, then a bad estimator that always outputs `[0, 0, ..., 0, 1]` can also be considered a probability distribution, right?  
  > **username_1**: @username_6, Yes to both.  The properties of softmax ensure that the output values represent a probability distribution, i.e., each number is in the range [0,1] and they sum to 1.  Yes, [0,0,...,0,1] is a probability distribution.  As you say, it might be a terrible estimate of the $p(y|x)$, but it is a probability distribution.  
  
  ---
  
  ### Answer by username_2 on 2022-11-14. Score: 12
  
  Excellent question.
  
  The simple answer is **no**. Softmax actually produces uncalibrated probabilities. That is, they do not really represent the probability of a prediction being correct.
  
  What usually happens is that softmax probabilities for the predicted class are closer to 100% in all cases, whether the predictions are correct or incorrect, which effectively does not give you any information. This is called overconfidence.
  
  This means that the probabilities are not useful, and you cannot really use them as reliable confidences to detect when the model is unsure or predicts incorrectly.
  
  For reference: Guo C, Pleiss G, Sun Y, Weinberger KQ. [On calibration of modern neural networks](http://proceedings.mlr.press/v70/guo17a.html). In International conference on machine learning 2017 Jul 17 (pp. 1321-1330). PMLR.
  
  > **Comments:**
  > **username_3**: Related question: If I include several copies of each image in my training set, and label some copies to some classes, and other copies to other classes, should I expect the softmax output to be close to the distributions in the training set? For instance, if a particular image has 4 copies in the training set, with 3 copies in class A and 1 copy in class B, and I run the classifier on this image, should I expect the output on that image to be close to 75% for A and 25% for B?  
  > **username_2**: @username_3 This will surely confuse the model but I do not think there is something generalizable about what would happen, it depends on the model, so you should not have an expectation. Label smoothing or mixup training would have the effect you mention.  
  > **username_4**: I'd like to point out that the original post contains 2 questions, in my view, and you're answering the second one correctly. First, we must say that, by the definition of a probability or probability vector, the softmax produces a probability vector and the sigmoid produces a probability (number between 0 and 1). Here, you answer the more relevant (in my view) question: do these mentioned probabilities represent good/true confidence/uncertainty values?  
  
  ---
  
  ### Answer by username_5 on 2024-12-25. Score: 1
  
  **Yes**, but as the other answers said, the raw softmax output has the issue of **overconfidence**. So, you need a **transformation** to make them linear like **logarithmic scale**.
  
  Logarithms have a unique relationship with infinity. As numbers approach infinity, logarithms grow, but they do so more slowly than the numbers themselves. This slow growth can be philosophically interpreted as a metaphor for how small actions or small steps can still have an infinite impact over time. **The logarithm provides a bridge between the finite (real-world measurements like IEEE 754 representation) and the infinite (theoretical concepts)**, suggesting that even seemingly small, finite changes can have profound, infinite consequences.
  
  To address the overconfidence issue, let $a$ be the **maximum** value of probability. Intuitively, putting them inside a logarithmic scale $ -\log\_2(1 - a)$ would reduce the overconfidence issue as it's sensitive to the small difference when the $a$ close to $1$, therefore it can be used for **confidence** measurement.
  
  Additionally, we can also consider the **minimum** value of probability to represent how assertive the softmax output's statement is as the support, and so that I can consider it as an **uncertainty** measurement.
  
  ---
  
  I tested it when I trained MNIST Handwritten Digit of 10 classes as usual. Where I use the original input and its 90-degree-rotated (transposed) input. Subjectively speaking, there is no such number that looks like a horizontal line for the class of digit number 1 when it's rotated while the class of digit number 0 is similar like its original when it's rotated.
  
  [![enter image description here](https://i.sstatic.net/lGPME7A9.png)](https://i.sstatic.net/lGPME7A9.png)
  
  ---
  
  Performing a transformation of raw softmax output into a logarithmic scale would produce probabilities of *I don't know* vs. *I know*, like this class of digit number 1 below. Notice that there are **gaps of numerical precision**, as the most real number is represented by IEEE 754 in most computers, which has limited precision.
  
  [![enter image description here](https://i.sstatic.net/jOKRqFdJ.png)](https://i.sstatic.net/jOKRqFdJ.png)
  
  ---
  
  Same for the class of digit number 0 below as expected, it is mostly correct to predict its rotated version as it's similar to the original.
  
  [![enter image description here](https://i.sstatic.net/ykDh9bf0.png)](https://i.sstatic.net/ykDh9bf0.png)
  
  ---
  
  Here is the full formula:
  
  ### Inputs:
  
  * Let $\mathbf{p} \in \mathbb{R}^{n\_{\text{samples}} \times n\_{\text{classes}}}$ be the input matrix of predicted probabilities, where:
    + $n\_{\text{samples}}$ is the number of samples.
    + $n\_{\text{classes}}$ is the number of classes (i.e., the number of possible predictions per sample).
    + Each element $p\_{i,j} \in [0, 1]$ represents the predicted probability for sample $i$ and class $j$, and for each $i$, we have $\sum\_{j=1}^{n\_{\text{classes}}} p\_{i,j} = 1$.
  
  ---
  
  ### Definitions:
  
  * Let $N = n\_{\text{classes}}$ denote the number of classes.
  * Let $\text{inf} = -\log\_2(\epsilon)$ (where $\epsilon$ is the smallest representable number for the input's data type)
  * For each sample $i$, define:
    + The maximum predicted probability $a\_i = \max\_j (p\_{i,j})$.
    + The minimum predicted probability $b\_i = \min\_j (p\_{i,j})$.
  
  ---
  
  ### Confidence:
  
  For each sample $i$, the confidence is defined as:
  
  $$
  c\_i = \log\_2(N - 1) - \log\_2(N) - \log\_2(1 - a\_i)
  $$
  
  where $a\_i = \max\_j (p\_{i,j})$ is the maximum predicted probability for sample $i$.
  
  **Specifically:**
  The confidence for each sample is normalized as:
  
  $$
  c\_i' = \frac{c\_i}{\inf}
  $$
  
  where $\inf$ is a large constant, representing the inverse of the smallest representable number of the data type, ensuring that the confidence is mapped to the range $[0, 1]$.
  
  ---
  
  ### Uncertainty:
  
  The uncertainty for each sample $i$ is defined as:
  
  $$
  u\_i = -\log\_2(N) - \log\_2(b\_i) - c\_i
  $$
  
  where $b\_i = \min\_j (p\_{i,j})$ is the minimum predicted probability for sample $i$, and $c\_i$ is the confidence for sample $i$.
  
  **Specifically:**
  The uncertainty for each sample is normalized as:
  
  $$
  u\_i' = \frac{u\_i}{\inf}
  $$
  
  where $\inf$ is the same large constant used for normalizing the confidence.
  
  Then, the uncertainty is transformed as:
  
  $$
  u\_i'' = \frac{2^{u\_i'} - 1}{2^{u\_i'} + 1}
  $$
  
  This maps the uncertainty values to the range $[0, 1]$.
  
  ---
  
  ### Outputs:
  
  * The output consists of two arrays:
    1. **Confidence array $\mathbf{c}$**: A normalized confidence value for each sample, where $c\_i' \in [0, 1]$.
    2. **Uncertainty array $\mathbf{u}$**: A normalized uncertainty value for each sample, where $u\_i'' \in [0, 1]$.
  
  The final output is a matrix $\mathbf{output} \in \mathbb{R}^{n\_{\text{samples}} \times 2}$, where each row contains the normalized confidence and uncertainty values for each sample.
  
  ---
  
  # Summary
  
  So, you can say that the softmax output can represent the probability, except it represents the probability of *I don't know this sample vs. I know this sample*.
  
  
  ---
  
  