Answers.Body: <p>Traditional neural networks can be <strong>over-confident</strong> (i.e. give a probability close to <span class="math-container">$0$</span> or <span class="math-container">$1$</span>) even when they are wrong, so you should <strong>not</strong> interpret the probability that it produces as a measure of <em>uncertainty</em> (i.e. as a measure of how much it is confident that the associated predicted class is the correct one), as that it is essentially wrong. See <a href="https://ai.stackexchange.com/a/17671/2444">this</a> and <a href="https://stats.stackexchange.com/a/437769/82135">this</a> answers for more details about this.</p>
  <p>Given that this overconfidence is not desirable in many scenarios (such as healthcare, where doctors also want to know how <em>confident</em> the model is about its predictions, in order to decide whether to give a certain medication to the patient or not), the ML community has been trying to incorporate <em>uncertainty quantification/estimation</em> in neural networks. If you are interested in this topic, you could read the paper <a href="http://proceedings.mlr.press/v37/blundell15.html" rel="nofollow noreferrer">Weight Uncertainty in Neural Network</a> (2015) by Blundell et al., which proposes a specific type of Bayesian neural network, i.e. a neural network that models the uncertainty over the actual values of the weights, from which we may also quantify/estimate the uncertainty about the inputs. This paper should not be too difficult to read if you are already familiar with the details of variational-autoencoders.</p>
  <p>So, the answer to your question is: yes, it's possible that the output probability is close to <span class="math-container">$1$</span> because neural networks can be over-confident. (I am assuming that the values returned by <code>tf.keras</code>'s <code>predict</code> method are probabilities: I don't remember anymore, so I assumed that you did not make any mistake).</p>
  <p>A similar question was already asked in the past <a href="https://ai.stackexchange.com/q/17721/2444">here</a>. <a href="https://ai.stackexchange.com/a/17722/2444">The accepted answer</a> should provide more details about different types of uncertainty and solutions.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-28T02:20:45.130Z
Answers.LastActivityDate: 2020-11-29T19:07:57.833Z
Answers.LastEditDate: 2020-11-29T19:07:57.833Z
Answers.UserDisplayName: nbro
Answers.Body: <p>Without more details about the nature of the dataset, it is impossible to know for sure. However, here are a few likely causes:</p>
  <ol>
  <li><p>You were calling predict on training data, not testing data. The network will be a lot more sure about images that it trained on than on images it has never seen before.</p>
  </li>
  <li><p>Your model overfit the data. This can happen when you use an overly complex model on a small dataset. You may want to experiment with regularization.</p>
  </li>
  <li><p>You were looking at too small a sample of images. Did you run predict on every image, or just a few? If the latter, it is possible you just picked a sample that the network is very confident about.</p>
  </li>
  </ol>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-26T23:13:31.547Z
Answers.LastActivityDate: 2020-11-26T23:13:31.547Z
Answers.UserDisplayName: chessprogrammer
Body: <p>I am using Keras (on top of TF 2.3) to train an image classifier. In some cases I have more than two classes, but often there are just two classes (either &quot;good&quot; or &quot;bad&quot;). I am using the <code>tensorflow.keras.applications.VGG16</code> class as base model with a custom classifier on top, like this:</p>
  <pre><code>input_layer = layers.Input(shape=(self.image_size, self.image_size, 3), name=&quot;model_input&quot;)
  base_model = VGG16(weights=&quot;imagenet&quot;, include_top=False, input_tensor=input_layer)
  model_head = base_model.output
  model_head = layers.AveragePooling2D(pool_size=(4, 4))(model_head)
  model_head = layers.Flatten()(model_head)
  model_head = layers.Dense(256, activation=&quot;relu&quot;)(model_head)
  model_head = layers.Dropout(0.5)(model_head)
  model_head = layers.Dense(len(self.image_classes), activation=&quot;softmax&quot;)(model_head)
  </code></pre>
  <p>As you can see in the last (output) layer I am using a <code>softmax</code> activation function. Then I compile the whole model with the <code>categorical_crossentropy</code> loss function and train with one-hot-encoded image data (labels).</p>
  <p>All in all the model performs quite well, I am happy with the results, I achieve over 99% test and validation accuracy with our data set. There is one thing I don't understand though:</p>
  <p>When I call <code>predict()</code> on the Keras model and look at the prediction results, then these are always either 0 or 1 (or at least very, very close to that, like 0.000001 and 0.999999). So my classifier seems to be quite sure whether an image belongs to either class &quot;good&quot; or &quot;bad&quot; (for example, if I am using only two classes). I was under the assumption, however, that usually these predictions are not that clear, more in terms of <em>&quot;the model thinks with a probability of 80% that this image belongs to class A&quot;</em> - but as said in my case it's always 100% sure.</p>
  <p>Any ideas why this might be the case?</p>
  
Tags: <deep-learning><computer-vision><keras><bayesian-deep-learning><uncertainty-quantification>
Title: Why is my Keras prediction always close to 100% for one image class?
UserDisplayName: Matthias
ThreadText: # Why is my Keras prediction always close to 100% for one image class?
  
  (Asked by: username_0 on 2020-11-26)
  
  I am using Keras (on top of TF 2.3) to train an image classifier. In some cases I have more than two classes, but often there are just two classes (either "good" or "bad"). I am using the `tensorflow.keras.applications.VGG16` class as base model with a custom classifier on top, like this:
  
  ```
  input_layer = layers.Input(shape=(self.image_size, self.image_size, 3), name="model_input")
  base_model = VGG16(weights="imagenet", include_top=False, input_tensor=input_layer)
  model_head = base_model.output
  model_head = layers.AveragePooling2D(pool_size=(4, 4))(model_head)
  model_head = layers.Flatten()(model_head)
  model_head = layers.Dense(256, activation="relu")(model_head)
  model_head = layers.Dropout(0.5)(model_head)
  model_head = layers.Dense(len(self.image_classes), activation="softmax")(model_head)
  
  ```
  
  As you can see in the last (output) layer I am using a `softmax` activation function. Then I compile the whole model with the `categorical_crossentropy` loss function and train with one-hot-encoded image data (labels).
  
  All in all the model performs quite well, I am happy with the results, I achieve over 99% test and validation accuracy with our data set. There is one thing I don't understand though:
  
  When I call `predict()` on the Keras model and look at the prediction results, then these are always either 0 or 1 (or at least very, very close to that, like 0.000001 and 0.999999). So my classifier seems to be quite sure whether an image belongs to either class "good" or "bad" (for example, if I am using only two classes). I was under the assumption, however, that usually these predictions are not that clear, more in terms of *"the model thinks with a probability of 80% that this image belongs to class A"* - but as said in my case it's always 100% sure.
  
  Any ideas why this might be the case?
  
  **Tags:** deep-learning, computer-vision, keras, bayesian-deep-learning, uncertainty-quantification
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2020-11-28. Score: 2 | Accepted answer
  
  Traditional neural networks can be **over-confident** (i.e. give a probability close to $0$ or $1$) even when they are wrong, so you should **not** interpret the probability that it produces as a measure of *uncertainty* (i.e. as a measure of how much it is confident that the associated predicted class is the correct one), as that it is essentially wrong. See [this](https://ai.stackexchange.com/a/17671/2444) and [this](https://stats.stackexchange.com/a/437769/82135) answers for more details about this.
  
  Given that this overconfidence is not desirable in many scenarios (such as healthcare, where doctors also want to know how *confident* the model is about its predictions, in order to decide whether to give a certain medication to the patient or not), the ML community has been trying to incorporate *uncertainty quantification/estimation* in neural networks. If you are interested in this topic, you could read the paper [Weight Uncertainty in Neural Network](http://proceedings.mlr.press/v37/blundell15.html) (2015) by Blundell et al., which proposes a specific type of Bayesian neural network, i.e. a neural network that models the uncertainty over the actual values of the weights, from which we may also quantify/estimate the uncertainty about the inputs. This paper should not be too difficult to read if you are already familiar with the details of variational-autoencoders.
  
  So, the answer to your question is: yes, it's possible that the output probability is close to $1$ because neural networks can be over-confident. (I am assuming that the values returned by `tf.keras`'s `predict` method are probabilities: I don't remember anymore, so I assumed that you did not make any mistake).
  
  A similar question was already asked in the past [here](https://ai.stackexchange.com/q/17721/2444). [The accepted answer](https://ai.stackexchange.com/a/17722/2444) should provide more details about different types of uncertainty and solutions.
  
  
  ---
  
  ### Answer by username_2 on 2020-11-26. Score: 1
  
  Without more details about the nature of the dataset, it is impossible to know for sure. However, here are a few likely causes:
  
  1. You were calling predict on training data, not testing data. The network will be a lot more sure about images that it trained on than on images it has never seen before.
  2. Your model overfit the data. This can happen when you use an overly complex model on a small dataset. You may want to experiment with regularization.
  3. You were looking at too small a sample of images. Did you run predict on every image, or just a few? If the latter, it is possible you just picked a sample that the network is very confident about.
  
  
  ---
  
  