Answers.Body: <p>Here are some points I noticed:</p>
  <ol>
  <li><p>Your data isn't enough for training a deep learning model from scratch. Like you mentioned using a pre-trained model is probably a better alternative like <a href="https://keras.io/api/applications/vgg/" rel="nofollow noreferrer">vgg</a></p>
  </li>
  <li><p>150 epochs is too much for the amount of data you have it's bound to overfit (however this isn't what will fix your problem here but something to consider for the future)</p>
  </li>
  <li><p>Try augmenting your data. In other words, you can increase your dataset size by randomly rotating images, flipping them, etc. check this <a href="https://www.tensorflow.org/tutorials/images/data_augmentation" rel="nofollow noreferrer">link out to learn more</a></p>
  </li>
  <li><p>Try out smaller learning rates maybe this is why your model isn't learning e.g try out lr=1e-3 or even less to see if that impacts your learning rate. Generally try to play around with your hyperparameters like batch size etc</p>
  </li>
  <li><p>If you didn't preprocess your images you should e.g turning them to grayscale, normalise the data (values ranging from 0 to 1 instead of 0 to 255), or maybe apply other image transformations to denoise your data and make it easier for your model to extract valuable information</p>
  </li>
  <li><p>Try using dropout layers and try playing around with the number units in your dense layers and filters in your convolutional layers. This seems to me to be more of an issue with your data more than anything but you can always play around with your model</p>
  </li>
  </ol>
  <p>Those are just some ideas. Deep learning models are really only as good as their training data so you probably should look more into that. Here's another <a href="https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html" rel="nofollow noreferrer">link</a> you can check out to see how to work with small datasets like yours. Another useful approach would be using <a href="https://keras.io/examples/vision/grad_cam/" rel="nofollow noreferrer">grad-cam</a>. This allows to see what your model considers from your input image to make its prediction so you can understand it better.</p>
  <p>Hope this helps.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-08T18:44:19.553Z
Answers.Text: Headless like vgg (or a more modern cousin) is a solid start.
Answers.UserDisplayName: EngrStudent
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-08T13:04:09.557Z
Answers.LastActivityDate: 2023-01-08T13:04:09.557Z
Answers.UserDisplayName: kimo26
Body: <p>So I am trying to make a CNN image classifier that has two classes, good and bad. The aim is to look at photoshoot pictures that can be found on fashion sites and find the &quot;best one&quot;. I trained this model for 150 epochs and its loss did not change at all(Roughly). Details are as follows:</p>
  <p>My metric for &quot;best one&quot;, also the way I structured my dataset, is that the photo shows the whole outfit or whole body of the model not only like upper body or lower. I also labeled the photos where the model's back was turned to the camera. My training set has 1304 good photos and 2000 bad photos. My validation set has 300 photos per class.(so 600)</p>
  <p>Architechture as follows : Conv &gt; Pool &gt; Conv &gt; Pool &gt; Flatten &gt; Linear &gt; Linear &gt; Softmax. For details of architecture like stride etc. check out the code I provided.</p>
  <p>I have to use softmax since in my application I need to see the probabilities of being good and bad. That is why I am not using cross-entropy loss but instead using negative log-likelihood loss. Adam is my optimizer</p>
  <p>Other hyperparameters: batch size: 64, number of epochs: 150, input size: (224, 224), number of classes: 2, learning rate: 0.01, weight decay: 0.01</p>
  <p>I trained with <a href="https://wtools.io/paste-code/bIBl" rel="nofollow noreferrer">this script</a> for 150 epochs. The model initialized with a 0.5253 loss and ended with a 0.5324 loss. I took snapshots every 10 epochs but I did not learn anything through the learning. This is what my learning curve looks like:</p>
  <p><a href="https://i.sstatic.net/ASagU.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/ASagU.png" alt="Learning Curve" /></a></p>
  <p>Now I know that there are many many things I can do to make the model perform better like initializing with a pretrained, doing some more stuff with transforms, etc. But the problem with this model is not that it is performing poorly, it is not performing at all! Like, I also have a validation accuracy check and it is around %50 during all training, for a classifier with 2 classes. So I am assuming I am doing something very obvious wrong, any idea what?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-02-08T18:42:03.917Z
Comments.Text: I would put the photos in a generator (like the one used in pix2pix) that randomly subsets and clips.  If you are going from 2560x2560 to 256x256 that improves your sample count.  This also means you can train on smaller and more meaningful subsets.  It also gives the learner smaller bites to chew through so it can sometimes improve progress, assuming progress is possible.  I always baseline on a simpler but capable tool.  (https://ai.googleblog.com/2022/10/do-modern-imagenet-classifiers.html; https://www.tensorflow.org/hub/tutorials/image_feature_vector)
Comments.UserDisplayName: EngrStudent
Tags: <convolutional-neural-networks><classification><image-recognition><softmax>
Title: Why didn't my convolutional image classifier network learn anything?
UserDisplayName: isa türk
ThreadText: # Why didn't my convolutional image classifier network learn anything?
  
  (Asked by: username_0 on 2023-01-07)
  
  So I am trying to make a CNN image classifier that has two classes, good and bad. The aim is to look at photoshoot pictures that can be found on fashion sites and find the "best one". I trained this model for 150 epochs and its loss did not change at all(Roughly). Details are as follows:
  
  My metric for "best one", also the way I structured my dataset, is that the photo shows the whole outfit or whole body of the model not only like upper body or lower. I also labeled the photos where the model's back was turned to the camera. My training set has 1304 good photos and 2000 bad photos. My validation set has 300 photos per class.(so 600)
  
  Architechture as follows : Conv > Pool > Conv > Pool > Flatten > Linear > Linear > Softmax. For details of architecture like stride etc. check out the code I provided.
  
  I have to use softmax since in my application I need to see the probabilities of being good and bad. That is why I am not using cross-entropy loss but instead using negative log-likelihood loss. Adam is my optimizer
  
  Other hyperparameters: batch size: 64, number of epochs: 150, input size: (224, 224), number of classes: 2, learning rate: 0.01, weight decay: 0.01
  
  I trained with [this script](https://wtools.io/paste-code/bIBl) for 150 epochs. The model initialized with a 0.5253 loss and ended with a 0.5324 loss. I took snapshots every 10 epochs but I did not learn anything through the learning. This is what my learning curve looks like:
  
  [![Learning Curve](https://i.sstatic.net/ASagU.png)](https://i.sstatic.net/ASagU.png)
  
  Now I know that there are many many things I can do to make the model perform better like initializing with a pretrained, doing some more stuff with transforms, etc. But the problem with this model is not that it is performing poorly, it is not performing at all! Like, I also have a validation accuracy check and it is around %50 during all training, for a classifier with 2 classes. So I am assuming I am doing something very obvious wrong, any idea what?
  
  **Tags:** convolutional-neural-networks, classification, image-recognition, softmax
  > **Comments:**
  > **username_1**: I would put the photos in a generator (like the one used in pix2pix) that randomly subsets and clips.  If you are going from 2560x2560 to 256x256 that improves your sample count.  This also means you can train on smaller and more meaningful subsets.  It also gives the learner smaller bites to chew through so it can sometimes improve progress, assuming progress is possible.  I always baseline on a simpler but capable tool.  (https://ai.googleblog.com/2022/10/do-modern-imagenet-classifiers.html; https://www.tensorflow.org/hub/tutorials/image_feature_vector)  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2023-01-08. Score: 1
  
  Here are some points I noticed:
  
  1. Your data isn't enough for training a deep learning model from scratch. Like you mentioned using a pre-trained model is probably a better alternative like [vgg](https://keras.io/api/applications/vgg/)
  2. 150 epochs is too much for the amount of data you have it's bound to overfit (however this isn't what will fix your problem here but something to consider for the future)
  3. Try augmenting your data. In other words, you can increase your dataset size by randomly rotating images, flipping them, etc. check this [link out to learn more](https://www.tensorflow.org/tutorials/images/data_augmentation)
  4. Try out smaller learning rates maybe this is why your model isn't learning e.g try out lr=1e-3 or even less to see if that impacts your learning rate. Generally try to play around with your hyperparameters like batch size etc
  5. If you didn't preprocess your images you should e.g turning them to grayscale, normalise the data (values ranging from 0 to 1 instead of 0 to 255), or maybe apply other image transformations to denoise your data and make it easier for your model to extract valuable information
  6. Try using dropout layers and try playing around with the number units in your dense layers and filters in your convolutional layers. This seems to me to be more of an issue with your data more than anything but you can always play around with your model
  
  Those are just some ideas. Deep learning models are really only as good as their training data so you probably should look more into that. Here's another [link](https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html) you can check out to see how to work with small datasets like yours. Another useful approach would be using [grad-cam](https://keras.io/examples/vision/grad_cam/). This allows to see what your model considers from your input image to make its prediction so you can understand it better.
  
  Hope this helps.
  
  > **Comments:**
  > **username_1**: Headless like vgg (or a more modern cousin) is a solid start.  
  
  ---
  
  