Answers.Body: <p>I feel your problem might not be with the model itself but with the dataset. If you only have <span class="math-container">$2500$</span> images for <span class="math-container">$24$</span> labels (hand-signs) that gets you roughly <span class="math-container">$104$</span> images per label. This is very little for the models I train (~<span class="math-container">$80K$</span> images in the smallest of cases). In my view you got a really decent accuracy at validation and test time for the size of your dataset.</p>
  
  <p>But answering your questions:</p>
  
  <ul>
  <li>Starting from few labels and extending is usually helpful when your model is too deep and suffers from convergence problems. Your model is simple enough not to suffer those problems so I would go learning the <span class="math-container">$24$</span> labels at once.</li>
  <li>Transfer learning can help a lot at reducing training times. For example, if you start from a <em>VGG</em> classifier that detects hands there is a good chance that the weights of your convolutional layers are already almost configured for your use case.</li>
  </ul>
  
  <p>Generally speaking, the easiest way to increase the accuracy of your model is to use one of these 2 methods:</p>
  
  <ul>
  <li>Increase the dataset: I am not sure if it is possible in your case, maybe you can use image augmentation (rotation, zooming in/out, changes in the color space...).</li>
  <li>Increase the depth of your model (provided that you have a big dataset).</li>
  </ul>
  
  <p>If you already fulfilled those items, then you can go and make changes to the model architecture or loss function. Looking at your model and from the top of my head I would try to add <em>Batch-Normalization</em> for the convolutional layers.</p>
  
  <p>Hope this helps a bit :)</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-13T09:53:33.400Z
Answers.Text: Thanks. I am actually having 2500 images for each hand-sign (each label). So total images are around 60000. I have used data augmentation before training the above model. I will try VGG16 classifier like:
  model = Sequential([  
  vgg16_base,  
  Dropout(0.2),
  Flatten(),
  Dropout(0.2),
  Dense(512, activation='relu', kernel_regularizer=l2(0.01)),
  Dense(NUM_CLASSES, activation='softmax')  
  ])  
  Also, I would unfreeze the last one/two layers. But should I add any other Conv2D, Maxpool, Flatten, or Dense Layers at the end?
Answers.UserDisplayName: mayuresh_sa
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-13T11:19:33.030Z
Answers.Text: Ah really nice then! 60K images is ok. I feel you could train VGG starting from some pretrained weights (for ILSVRC for example) instead of freezing layers. Regarding adding layers, you should add Conv2D in the Conv2D  stage before flattening to the Dense stage (same with Dense, add them only in the Dense layers stage). And yes, you could make your model deeper since you have enough training data that normally boost performance because it allows to learn more complex nuances of the use case.
Answers.UserDisplayName: JVGD
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-12T13:45:13.400Z
Answers.LastActivityDate: 2020-02-12T13:51:40.557Z
Answers.LastEditDate: 2020-02-12T13:51:40.557Z
Answers.UserDisplayName: JVGD
Body: <p>I am developing a CNN model to recognize 24 hand-signs of American Sign Language. I have 2500 Images/hand-sign. The data split is:<br />
  Training = 1250 Images/hand-sign<br />
  Validation = 625 Images/hand-sign<br />
  Testing = 625 Images/hand-sign<br /></p>
  
  <p>How should I proceed with training the model?:<br />
  1. Should I develop a model starting from fewer hand-signs (like 5) and then increase them gradually?<br />
  2. Should I start models from scratch or use transfer learning (VGG16 or other)<br />
  Applying data augmentation, I did some tests with VGG16 and added a dense classifier at the end and received these accuracies:<br />
  Train: 0.87610877<br />
  Validation: 0.8867307<br />
  Test: 0.96533334<br /></p>
  
  <p><a href="https://i.sstatic.net/lq11e.png" rel="nofollow noreferrer">Accuracy and Loss Graph</a></p>
  
  <p>Test parameters:<br />
  NUM_CLASSES = 5<br />
  EPOCHS = 50<br />
  STEPS_PER_EPOCH = 125<br />
  VALIDATION_STEPS = 75<br />
  TEST_STEPS = 75<br />
  Framework = Keras, Tensorflow<br />
  OPTIMIZER = adam<br /></p>
  
  <p>Model:</p>
  
  <pre><code>model = Sequential([
      Conv2D(32, (3, 3), activation='relu', input_shape=(IMG_HEIGHT, IMG_WIDTH ,3)),
      MaxPooling2D(pool_size=(2,2)),
  
      Conv2D(64, (3, 3), activation='relu'),
      MaxPooling2D(pool_size=(2,2)),
  
      Conv2D(128, (3, 3), activation='relu'),
      MaxPooling2D(pool_size=(2,2)),
  
      Conv2D(256, (3, 3), activation='relu'),
      MaxPooling2D(pool_size=(2,2)),
  
      Conv2D(512, (3, 3), activation='relu'),
      MaxPooling2D(pool_size=(2,2)),
  
      Flatten(),
      Dense(512, activation='relu'),
  
      Dense(NUM_CLASSES, activation='softmax')
  ])
  </code></pre>
  
  <p>If I try images with slightly different background and predict the classes (predict_classes()), I do not get accurate results. Any suggestions on how to make the model robust?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-13T14:22:44.537Z
Comments.Text: Please, next time, focus on a single problem and questions. For each of your questions, open a different thread/post.
Comments.UserDisplayName: nbro
Tags: <neural-networks><deep-learning><convolutional-neural-networks><classification><keras>
Title: Hand-Signs Recognition using Deep Learning Convolutional Neural Networks
UserDisplayName: mayuresh_sa
ThreadText: # Hand-Signs Recognition using Deep Learning Convolutional Neural Networks
  
  (Asked by: username_0 on 2020-02-11)
  
  I am developing a CNN model to recognize 24 hand-signs of American Sign Language. I have 2500 Images/hand-sign. The data split is:  
  Training = 1250 Images/hand-sign  
  Validation = 625 Images/hand-sign  
  Testing = 625 Images/hand-sign
  
  How should I proceed with training the model?:  
  1. Should I develop a model starting from fewer hand-signs (like 5) and then increase them gradually?  
  2. Should I start models from scratch or use transfer learning (VGG16 or other)  
  Applying data augmentation, I did some tests with VGG16 and added a dense classifier at the end and received these accuracies:  
  Train: 0.87610877  
  Validation: 0.8867307  
  Test: 0.96533334
  
  [Accuracy and Loss Graph](https://i.sstatic.net/lq11e.png)
  
  Test parameters:  
  NUM\_CLASSES = 5  
  EPOCHS = 50  
  STEPS\_PER\_EPOCH = 125  
  VALIDATION\_STEPS = 75  
  TEST\_STEPS = 75  
  Framework = Keras, Tensorflow  
  OPTIMIZER = adam
  
  Model:
  
  ```
  model = Sequential([
      Conv2D(32, (3, 3), activation='relu', input_shape=(IMG_HEIGHT, IMG_WIDTH ,3)),
      MaxPooling2D(pool_size=(2,2)),
  
      Conv2D(64, (3, 3), activation='relu'),
      MaxPooling2D(pool_size=(2,2)),
  
      Conv2D(128, (3, 3), activation='relu'),
      MaxPooling2D(pool_size=(2,2)),
  
      Conv2D(256, (3, 3), activation='relu'),
      MaxPooling2D(pool_size=(2,2)),
  
      Conv2D(512, (3, 3), activation='relu'),
      MaxPooling2D(pool_size=(2,2)),
  
      Flatten(),
      Dense(512, activation='relu'),
  
      Dense(NUM_CLASSES, activation='softmax')
  ])
  
  ```
  
  If I try images with slightly different background and predict the classes (predict\_classes()), I do not get accurate results. Any suggestions on how to make the model robust?
  
  **Tags:** neural-networks, deep-learning, convolutional-neural-networks, classification, keras
  
  > **Comments:**
  > **username_1**: Please, next time, focus on a single problem and questions. For each of your questions, open a different thread/post.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-02-12. Score: 0
  
  I feel your problem might not be with the model itself but with the dataset. If you only have $2500$ images for $24$ labels (hand-signs) that gets you roughly $104$ images per label. This is very little for the models I train (~$80K$ images in the smallest of cases). In my view you got a really decent accuracy at validation and test time for the size of your dataset.
  
  But answering your questions:
  
  * Starting from few labels and extending is usually helpful when your model is too deep and suffers from convergence problems. Your model is simple enough not to suffer those problems so I would go learning the $24$ labels at once.
  * Transfer learning can help a lot at reducing training times. For example, if you start from a *VGG* classifier that detects hands there is a good chance that the weights of your convolutional layers are already almost configured for your use case.
  
  Generally speaking, the easiest way to increase the accuracy of your model is to use one of these 2 methods:
  
  * Increase the dataset: I am not sure if it is possible in your case, maybe you can use image augmentation (rotation, zooming in/out, changes in the color space...).
  * Increase the depth of your model (provided that you have a big dataset).
  
  If you already fulfilled those items, then you can go and make changes to the model architecture or loss function. Looking at your model and from the top of my head I would try to add *Batch-Normalization* for the convolutional layers.
  
  Hope this helps a bit :)
  
  > **Comments:**
  > **username_0**: Thanks. I am actually having 2500 images for each hand-sign (each label). So total images are around 60000. I have used data augmentation before training the above model. I will try VGG16 classifier like: model = Sequential([   vgg16_base,   Dropout(0.2), Flatten(), Dropout(0.2), Dense(512, activation='relu', kernel_regularizer=l2(0.01)), Dense(NUM_CLASSES, activation='softmax')   ])   Also, I would unfreeze the last one/two layers. But should I add any other Conv2D, Maxpool, Flatten, or Dense Layers at the end?  
  > **username_2**: Ah really nice then! 60K images is ok. I feel you could train VGG starting from some pretrained weights (for ILSVRC for example) instead of freezing layers. Regarding adding layers, you should add Conv2D in the Conv2D  stage before flattening to the Dense stage (same with Dense, add them only in the Dense layers stage). And yes, you could make your model deeper since you have enough training data that normally boost performance because it allows to learn more complex nuances of the use case.  
  
  ---
  
  