Answers.Body: <p>Yes, due to this issue, you should use temperature scaling after training your model. It will calibrate your probability and you will start to get the same kind of distributions. Here are a good article along with implementation on <a href="https://towardsdatascience.com/neural-network-calibration-using-pytorch-c44b7221a61" rel="nofollow noreferrer">it</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-02-26T08:50:13.140Z
Answers.Text: Thanks! This is a useful answer and checks the "practical" box. I'm also wondering a lot about the data science behind why this is the case. I had previously thought that logistic regression (which is essentially what's happening on top of the CNN) is supposed to give calibrated probabilities. In fact, that's what [Platt Scaling](https://en.wikipedia.org/wiki/Platt_scaling) is really doing.
Answers.UserDisplayName: Alexander Soare
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-02-26T09:08:16.940Z
Answers.Text: The reason each time, you get a different model is because of the entropy involved in creating the new model (initialization). Hence, reproducibility has come into scene strongly in DL. But, even if we set the random seed at every possible level, some amount of entropy is still retained. It is that entropy which leads to a different solution every single time. A lot of regularization may help with that, one should experiment. So, every time, you end up with a different minima in the loss landscape.
Answers.UserDisplayName: Abhishek Verma
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-02-25T19:42:26.397Z
Answers.LastActivityDate: 2021-02-25T19:42:26.397Z
Answers.UserDisplayName: Abhishek Verma
Body: <p>I've been working on a lot of simple resnet18 binary classifiers lately and I've started to notice that the probability distributions are often skewed one way or the other. This figure shows one such example. The red and blue color code the negative and positive ground truths respectively. And the bottom axis is the output prediction of the binary classifier (sigmoid activated output neuron). Notice how the red is more bunched towards 0, but the blue has quite some spread.</p>
  <p><a href="https://i.sstatic.net/kkBaZ.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/kkBaZ.png" alt="enter image description here" /></a></p>
  <p>At first I began to reason this to myself with arguments like <em>&quot;well the positive clues in the image have a small footprint, so they are hard to find, therefore the model should be unsure about positives more of the time.&quot;</em></p>
  <p>Later I found oppositely skewed distributions and tried to say <em>&quot;well the positive clues in the image have a small footprint, so it might be easy to confuse some other things for the positive clues, therefore the model should be unsure about negatives more of the time&quot;</em></p>
  <p>You can see where I'm going with this. It took me training up quite a few models like this in a short amount of time to realise I was kidding myself. Even the exact same architecture and similar dataset may produce a different skew over different training runs. And if you think about it, negative probability is just the complement of positive probability, so any argument you make in favor of one over the other can be easily reversed.</p>
  <p>So what's influencing this skew? Why is there a skew at all? If there's a skew, is it because of something &quot;real&quot;, or is it just random?</p>
  <p>These all may seem like philosophical questions, but they have <strong>great practical significance</strong>. Because that skew basically tells me where I should put my decision threshold in production level inference!</p>
  
Tags: <convolutional-neural-networks><probability-distribution><explainable-ai><binary-classification>
Title: Why are CNN binary classifier output probability distributions often skewed?
UserDisplayName: Alexander Soare
ThreadText: # Why are CNN binary classifier output probability distributions often skewed?
  
  (Asked by: username_0 on 2021-02-25)
  
  I've been working on a lot of simple resnet18 binary classifiers lately and I've started to notice that the probability distributions are often skewed one way or the other. This figure shows one such example. The red and blue color code the negative and positive ground truths respectively. And the bottom axis is the output prediction of the binary classifier (sigmoid activated output neuron). Notice how the red is more bunched towards 0, but the blue has quite some spread.
  
  [![enter image description here](https://i.sstatic.net/kkBaZ.png)](https://i.sstatic.net/kkBaZ.png)
  
  At first I began to reason this to myself with arguments like *"well the positive clues in the image have a small footprint, so they are hard to find, therefore the model should be unsure about positives more of the time."*
  
  Later I found oppositely skewed distributions and tried to say *"well the positive clues in the image have a small footprint, so it might be easy to confuse some other things for the positive clues, therefore the model should be unsure about negatives more of the time"*
  
  You can see where I'm going with this. It took me training up quite a few models like this in a short amount of time to realise I was kidding myself. Even the exact same architecture and similar dataset may produce a different skew over different training runs. And if you think about it, negative probability is just the complement of positive probability, so any argument you make in favor of one over the other can be easily reversed.
  
  So what's influencing this skew? Why is there a skew at all? If there's a skew, is it because of something "real", or is it just random?
  
  These all may seem like philosophical questions, but they have **great practical significance**. Because that skew basically tells me where I should put my decision threshold in production level inference!
  
  **Tags:** convolutional-neural-networks, probability-distribution, explainable-ai, binary-classification
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-02-25. Score: 1
  
  Yes, due to this issue, you should use temperature scaling after training your model. It will calibrate your probability and you will start to get the same kind of distributions. Here are a good article along with implementation on [it](https://towardsdatascience.com/neural-network-calibration-using-pytorch-c44b7221a61).
  
  > **Comments:**
  > **username_0**: Thanks! This is a useful answer and checks the "practical" box. I'm also wondering a lot about the data science behind why this is the case. I had previously thought that logistic regression (which is essentially what's happening on top of the CNN) is supposed to give calibrated probabilities. In fact, that's what [Platt Scaling](https://en.wikipedia.org/wiki/Platt_scaling) is really doing.  
  > **username_1**: The reason each time, you get a different model is because of the entropy involved in creating the new model (initialization). Hence, reproducibility has come into scene strongly in DL. But, even if we set the random seed at every possible level, some amount of entropy is still retained. It is that entropy which leads to a different solution every single time. A lot of regularization may help with that, one should experiment. So, every time, you end up with a different minima in the loss landscape.  
  
  ---
  
  