Answers.Body: <p>The answer (<a href="https://stackoverflow.com/questions/42081257/keras-binary-crossentropy-vs-categorical-crossentropy-performance">as detailed quite thoroughly here</a> and <a href="https://stackoverflow.com/questions/45799474/keras-model-evaluate-vs-model-predict-accuracy-difference-in-multi-class-nlp-ta/45834857#45834857">here</a>) is that specifying</p>
  
  <pre><code>model.compile(optimizer = 'rmsprop', loss = 'binary_crossentropy', metrics=['accuracy'])
  </code></pre>
  
  <p>causes keras to guess, incorrectly, that because I am using <code>binary_crossentropy</code> for the loss function, that I would want to use <code>binary_accuracy</code> as the way of reporting the accuracy metrics. Apparently, one should specify that one wants the <code>categorical_accuracy</code> metrics if one, as I do, has more than two classes.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-03-18T21:52:41.733Z
Answers.LastActivityDate: 2019-03-18T23:04:02.620Z
Answers.LastEditDate: 2019-03-18T23:04:02.620Z
Answers.UserDisplayName: pr3sidentspence
Body: <p>I am attempting to train a network to do something I thought would be a relatively simple case to learn with: identify whether the back of a scanned vintage postcard has one of 'no postage stamp', a '1 cent stamp', or a '2 cent stamp.' The images are 250px by about 150px, RGB color, and there are about two thousand of them. Ballpark 75% of them are no-stamp, 20% 1-cent, and 10% 2-cent.</p>
  
  <p>When I attempt to train the network it seems like it is starting at 70 +/- 1 % accurate and hovers in that range for 50 epochs, never improving. I'm not sure I'm reading the metrics correctly, though, as this doesn't seem quite right.</p>
  
  <p>I set this up by following the tutorial on the Keras blog: <a href="https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html" rel="nofollow noreferrer">https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html</a> </p>
  
  <p>I haven't implemented the latter part of the tutorial, where a pre-trained network is used, because I haven't found one that seems like it would be a similar problem.</p>
  
  <p>My training and validation sets are here: <a href="https://drive.google.com/open?id=1-TxEKVVvP7RuFC7kFgH7Wt5A8z8QGTR3" rel="nofollow noreferrer">https://drive.google.com/open?id=1-TxEKVVvP7RuFC7kFgH7Wt5A8z8QGTR3</a> </p>
  
  <p>And my Google Colab Jupyter notebook is here: <a href="https://colab.research.google.com/drive/1UuKDF1wDwYlXszB2ahIrygRnfcs2D_sD" rel="nofollow noreferrer">https://colab.research.google.com/drive/1UuKDF1wDwYlXszB2ahIrygRnfcs2D_sD</a> </p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-03-18T09:30:28.293Z
Comments.Text: Machine learning unfortunately doesn't necessarily get better with more training data. If the classes in your problem are not clearly separable for the algorithm, then no amount of data will improve it. One issue could be that the classes are unbalanced; perhaps try to use fewer 'no-stamp' images. Or investigate pre-processing them, eg to amplify colours or other features, so that the classifier can pick them up better.
Comments.UserDisplayName: Oliver Mason
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-03-18T14:47:04.643Z
Comments.Text: Thanks @OliverMason, I will try that. I was also thinking that maybe I need to make sure the ImageGenerator() isn't cropping the image such that the stamp area is missing.
Comments.UserDisplayName: pr3sidentspence
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-03-18T16:18:58.960Z
Comments.Text: I forgot that I already did that, re: ImageGenerator(). :\
Comments.UserDisplayName: pr3sidentspence
Tags: <convolutional-neural-networks><image-recognition><classification><keras>
Title: Why doesn't my image classification network get better with training?
UserDisplayName: pr3sidentspence
ThreadText: # Why doesn't my image classification network get better with training?
  
  (Asked by: username_0 on 2019-03-18)
  
  I am attempting to train a network to do something I thought would be a relatively simple case to learn with: identify whether the back of a scanned vintage postcard has one of 'no postage stamp', a '1 cent stamp', or a '2 cent stamp.' The images are 250px by about 150px, RGB color, and there are about two thousand of them. Ballpark 75% of them are no-stamp, 20% 1-cent, and 10% 2-cent.
  
  When I attempt to train the network it seems like it is starting at 70 +/- 1 % accurate and hovers in that range for 50 epochs, never improving. I'm not sure I'm reading the metrics correctly, though, as this doesn't seem quite right.
  
  I set this up by following the tutorial on the Keras blog: <https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html>
  
  I haven't implemented the latter part of the tutorial, where a pre-trained network is used, because I haven't found one that seems like it would be a similar problem.
  
  My training and validation sets are here: <https://drive.google.com/open?id=1-TxEKVVvP7RuFC7kFgH7Wt5A8z8QGTR3>
  
  And my Google Colab Jupyter notebook is here: <https://colab.research.google.com/drive/1UuKDF1wDwYlXszB2ahIrygRnfcs2D_sD>
  
  **Tags:** convolutional-neural-networks, image-recognition, classification, keras
  
  > **Comments:**
  > **username_1**: Machine learning unfortunately doesn't necessarily get better with more training data. If the classes in your problem are not clearly separable for the algorithm, then no amount of data will improve it. One issue could be that the classes are unbalanced; perhaps try to use fewer 'no-stamp' images. Or investigate pre-processing them, eg to amplify colours or other features, so that the classifier can pick them up better.  
  > **username_0**: Thanks @username_1, I will try that. I was also thinking that maybe I need to make sure the ImageGenerator() isn't cropping the image such that the stamp area is missing.  
  > **username_0**: I forgot that I already did that, re: ImageGenerator(). :\  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2019-03-18. Score: 1
  
  The answer ([as detailed quite thoroughly here](https://stackoverflow.com/questions/42081257/keras-binary-crossentropy-vs-categorical-crossentropy-performance) and [here](https://stackoverflow.com/questions/45799474/keras-model-evaluate-vs-model-predict-accuracy-difference-in-multi-class-nlp-ta/45834857#45834857)) is that specifying
  
  ```
  model.compile(optimizer = 'rmsprop', loss = 'binary_crossentropy', metrics=['accuracy'])
  
  ```
  
  causes keras to guess, incorrectly, that because I am using `binary_crossentropy` for the loss function, that I would want to use `binary_accuracy` as the way of reporting the accuracy metrics. Apparently, one should specify that one wants the `categorical_accuracy` metrics if one, as I do, has more than two classes.
  
  
  ---
  
  