Answers.Body: <p>Based on having 16 classes, the naïve accuracy of a dummy classifier would be around 1/16 (6.25%) assuming class balance and the dummy classifier predicts each class according to uniform probabilities (see scikit-learn article on <a href="https://scikit-learn.org/stable/modules/generated/sklearn.dummy.DummyClassifier.html" rel="nofollow noreferrer">dummy classifiers</a>).  Therefore, your model is likely learning something useful beyond baseline.</p>
  <p>Depending on the complexity of your images, your training set having only 750 images may be on the low side and may be the possible/likely culprit for low accuracy.  Computer vision applications often require thousands of images for training.  For example, see CIFAR-100, CIFAR-10, MNIST digits, and MNIST fashion datasets, which have thousands of images.  For these large datasets, the achievable accuracy decreases as the complexity of the images increases (compare MNIST digits vs. MNIST fashion) and as the number of classes increases (compare CIFAR-100 vs. CIFAR-10).</p>
  <p><strong>Here is what I suggest for diagnosis:</strong></p>
  <ol>
  <li><p>Plot training and validation loss and accuracy vs. epoch to assess whether your model is overfitting / underfitting.</p>
  </li>
  <li><p>Determine the accuracy achievable by humans on the same dataset to assess what type of accuracy may be reasonable.  Of course, AI can outperform or underperform humans, so take this with a grain of salt.</p>
  </li>
  <li><p>If this a public dataset, search the literature for what accuracy other investigators have achieved.</p>
  </li>
  </ol>
  <p><strong>Here is what I suggest to boost accuracy after you have run the diagnostics:</strong></p>
  <ol>
  <li><p>Consider generating more real data and/or using <a href="https://keras.io/guides/keras_cv/custom_image_augmentations/" rel="nofollow noreferrer">Data Augmentation</a>.</p>
  </li>
  <li><p>Tune hyperparameters based on whether you are overfitting or underfitting.</p>
  </li>
  <li><p>Consider using a pre-trained model.</p>
  </li>
  </ol>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-24T02:34:48.473Z
Answers.LastActivityDate: 2022-10-24T02:42:38.930Z
Answers.LastEditDate: 2022-10-24T02:42:38.930Z
Answers.UserDisplayName: Snehal Patel
Body: <p>I am trying to do multi class(16) classification, however no matter what parameters or number of layers I use my accuracy is not improving, its in 30s the max I got was 43.</p>
  <p>I have tried early stopping to red overfilling but my testing accuracy is still low.</p>
  <p>I have 750 images in training and 350 in testing. I am also getting high traning accuracy vs low validation accuracy.</p>
  <pre><code>features_train=features_train/255
  features_test=features_test/255
  
  cnn = models.Sequential([
      layers.Conv2D(filters=16, kernel_size=(3, 3), activation='relu',  strides=(2, 2), padding=&quot;same&quot;, input_shape=(224, 224, 3)),
  
      layers.MaxPooling2D((2,2)),
      layers.Dropout(0.25),
      layers.Conv2D(32 ,(3, 3), activation='relu'),
      layers.MaxPooling2D((2,2)),
      layers.Dropout(0.25),
      layers.Conv2D(64, (3, 3), activation='relu'),
      layers.MaxPooling2D((2,2)),
      layers.Dropout(0.25),
      layers.Conv2D(128, (3, 3), activation='relu'),
   
      layers.MaxPooling2D((2,2)),
      
      
      layers.Flatten(),
      layers.Dense(64,activation='relu'),
      layers.Dense(16, activation='softmax')
  ])
  cnn.compile(optimizer='adam',
               loss='categorical_crossentropy',
               metrics=['accuracy'])
  cnn.fit(features_train,labels_train,epochs=20, batch_size = 4 ,validation_split = 0.25)
  </code></pre>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-10-25T03:13:27.020Z
Comments.Text: Because you have a relatively small dataset, you should consider performing an n-fold crossvalidation rather than a holdout validation.  This will give you a better estimate of accuracy along with error bounds.
Comments.UserDisplayName: Snehal Patel
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-11-18T09:25:59.897Z
Comments.Text: increase batch_size to 128 or something. batch size of 4 for learning_rate 0.001 (default keras param for Adam) will not converge.
Comments.UserDisplayName: Abhas Kumar Sinha
Tags: <convolutional-neural-networks><keras>
Title: keras model accuracy not improving
UserDisplayName: SACHI SINGH
ThreadText: # keras model accuracy not improving
  
  (Asked by: username_0 on 2022-09-10)
  
  I am trying to do multi class(16) classification, however no matter what parameters or number of layers I use my accuracy is not improving, its in 30s the max I got was 43.
  I have tried early stopping to red overfilling but my testing accuracy is still low.
  I have 750 images in training and 350 in testing. I am also getting high traning accuracy vs low validation accuracy.
  features_train=features_train/255
  features_test=features_test/255
  
  cnn = models.Sequential([
      layers.Conv2D(filters=16, kernel_size=(3, 3), activation='relu',  strides=(2, 2), padding="same", input_shape=(224, 224, 3)),
  
      layers.MaxPooling2D((2,2)),
      layers.Dropout(0.25),
      layers.Conv2D(32 ,(3, 3), activation='relu'),
      layers.MaxPooling2D((2,2)),
      layers.Dropout(0.25),
      layers.Conv2D(64, (3, 3), activation='relu'),
      layers.MaxPooling2D((2,2)),
      layers.Dropout(0.25),
      layers.Conv2D(128, (3, 3), activation='relu'),
   
      layers.MaxPooling2D((2,2)),
      
      
      layers.Flatten(),
      layers.Dense(64,activation='relu'),
      layers.Dense(16, activation='softmax')
  ])
  cnn.compile(optimizer='adam',
               loss='categorical_crossentropy',
               metrics=['accuracy'])
  cnn.fit(features_train,labels_train,epochs=20, batch_size = 4 ,validation_split = 0.25)
  
  **Tags:** convolutional-neural-networks, keras
  > **Comments:**
  > **username_1**: Because you have a relatively small dataset, you should consider performing an n-fold crossvalidation rather than a holdout validation.  This will give you a better estimate of accuracy along with error bounds.  
  > **username_2**: increase batch_size to 128 or something. batch size of 4 for learning_rate 0.001 (default keras param for Adam) will not converge.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-10-24. Score: 0
  
  Based on having 16 classes, the naïve accuracy of a dummy classifier would be around 1/16 (6.25%) assuming class balance and the dummy classifier predicts each class according to uniform probabilities (see scikit-learn article on [dummy classifiers](https://scikit-learn.org/stable/modules/generated/sklearn.dummy.DummyClassifier.html)). Therefore, your model is likely learning something useful beyond baseline.
  
  Depending on the complexity of your images, your training set having only 750 images may be on the low side and may be the possible/likely culprit for low accuracy. Computer vision applications often require thousands of images for training. For example, see CIFAR-100, CIFAR-10, MNIST digits, and MNIST fashion datasets, which have thousands of images. For these large datasets, the achievable accuracy decreases as the complexity of the images increases (compare MNIST digits vs. MNIST fashion) and as the number of classes increases (compare CIFAR-100 vs. CIFAR-10).
  
  **Here is what I suggest for diagnosis:**
  
  1. Plot training and validation loss and accuracy vs. epoch to assess whether your model is overfitting / underfitting.
  2. Determine the accuracy achievable by humans on the same dataset to assess what type of accuracy may be reasonable. Of course, AI can outperform or underperform humans, so take this with a grain of salt.
  3. If this a public dataset, search the literature for what accuracy other investigators have achieved.
  
  **Here is what I suggest to boost accuracy after you have run the diagnostics:**
  
  1. Consider generating more real data and/or using [Data Augmentation](https://keras.io/guides/keras_cv/custom_image_augmentations/).
  2. Tune hyperparameters based on whether you are overfitting or underfitting.
  3. Consider using a pre-trained model.
  
  
  ---
  
  