Answers.Body: <p>When you use sigmoid activation it is applied independently to all outputs so outputs won't sum up to 1. Sigmoid is usually used for binary logistic regression where you only have 2 classes. In your case you should use softmax activation at the output which will squash outputs to range [0,1] and additionally make them sum up to 1.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-04-08T14:41:06.667Z
Answers.LastActivityDate: 2019-04-08T14:41:06.667Z
Answers.UserDisplayName: Brale
Body: <p>I'm a bit of a CNN newbie, and I'm trying to train one to image classify pictures of pretty similar looking particles. I'm making the inputs and labels by hand from a set of 48x48 grayscale images, and labeling them with a one-hot vector based on their position in the sequence (for example, the 400/1000th image might have a one-hot in the 4th position if I have 10 categories in the run). I'm using sigmoidal output activation and categorical cross entropy loss. I've played around with a few different optimizers, as well. I'm implementing in python keras. </p>
  
  <p>Unfortunately, although I have pretty good accuracy numbers for the training and validation, when I actually look at the outputs being produced, it generally gives multiple categories, which is not at all what I want. For example, if I have 6 categories and a label of 3, it might give the following probability vector:</p>
  
  <p>[ .99 .98  1.0  .99  0.02  0.05 ]</p>
  
  <p>It was my understanding that categorical cross entropy would not allow this type of categorization, and yet it is prevalent in my code. I am under the impression that I'm doing something fundamentally wrong, but I cant figure out what. Any help would be appreciated. </p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-04-08T13:57:21.970Z
Comments.Text: when you use sigmoid activation it is applied independently to all outputs so outputs won't sum up to 1. Sigmoid is usually used for binary logistic regression where you only have 2 classes. In your case you should use softmax activation at the output which will squash outputs to range [0,1] and additionally make them sum up to 1.
Comments.UserDisplayName: Brale
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-04-08T14:18:35.167Z
Comments.Text: I knew it would be a rookie mistake! Thank you! That fixed that issue, at least. If you want to post this as a full answer I will mark it as a correct fix.
Comments.UserDisplayName: Fred E
Tags: <convolutional-neural-networks><image-recognition><training><categorical-data>
Title: CNN output generally has more than one category in one-hot categorization?
UserDisplayName: Fred E
ThreadText: # CNN output generally has more than one category in one-hot categorization?
  
  (Asked by: username_0 on 2019-04-08)
  
  I'm a bit of a CNN newbie, and I'm trying to train one to image classify pictures of pretty similar looking particles. I'm making the inputs and labels by hand from a set of 48x48 grayscale images, and labeling them with a one-hot vector based on their position in the sequence (for example, the 400/1000th image might have a one-hot in the 4th position if I have 10 categories in the run). I'm using sigmoidal output activation and categorical cross entropy loss. I've played around with a few different optimizers, as well. I'm implementing in python keras.
  
  Unfortunately, although I have pretty good accuracy numbers for the training and validation, when I actually look at the outputs being produced, it generally gives multiple categories, which is not at all what I want. For example, if I have 6 categories and a label of 3, it might give the following probability vector:
  
  [ .99 .98 1.0 .99 0.02 0.05 ]
  
  It was my understanding that categorical cross entropy would not allow this type of categorization, and yet it is prevalent in my code. I am under the impression that I'm doing something fundamentally wrong, but I cant figure out what. Any help would be appreciated.
  
  **Tags:** convolutional-neural-networks, image-recognition, training, categorical-data
  > **Comments:**
  > **username_1**: when you use sigmoid activation it is applied independently to all outputs so outputs won't sum up to 1. Sigmoid is usually used for binary logistic regression where you only have 2 classes. In your case you should use softmax activation at the output which will squash outputs to range [0,1] and additionally make them sum up to 1.  
  > **username_0**: I knew it would be a rookie mistake! Thank you! That fixed that issue, at least. If you want to post this as a full answer I will mark it as a correct fix.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-04-08. Score: 1 | Accepted answer
  
  When you use sigmoid activation it is applied independently to all outputs so outputs won't sum up to 1. Sigmoid is usually used for binary logistic regression where you only have 2 classes. In your case you should use softmax activation at the output which will squash outputs to range [0,1] and additionally make them sum up to 1.
  
  
  ---
  
  