Answers.Body: <p>First off, if you really have to use that sequential model of yours then add a <a href="https://www.tensorflow.org/api_docs/python/tf/keras/layers/BatchNormalization" rel="nofollow noreferrer">Batch Normalization layer</a> after each MaxPooling layer. Try getting rid of the kernel regularizer, using a lower dropout rate and GlobalAveragePooling instead of Flatten as well. Otherwise just use a well-known model such as the <a href="https://github.com/qubvel/efficientnet" rel="nofollow noreferrer">EfficientNet</a>. Start with the EfficientNetB0 and see if it overfits. If it doesn't, then go up the ladder (B1, B2,...).</p>
  <p>Second: use some sort of data augmentation. I recommend using the <a href="https://albumentations.ai/" rel="nofollow noreferrer">Albumentations</a> framework.</p>
  <p>Last but not least, you're using the wrong loss. If you have a softmax output, then you should be using sparse cross-entropy instead of categorical. Categorical cross-entropy means that the outputs are simultaneous (i.e. two different labels can happen at the same time), which is the opposite of what a <a href="https://en.wikipedia.org/wiki/Softmax_function#:%7E:text=The%20softmax%20function%2C%20also%20known,used%20in%20multinomial%20logistic%20regression." rel="nofollow noreferrer">softmax</a> implies (output is a probability distribution, i.e. the sum of the outputs is equal to 1). If you think that the output labels can happen at the same time and that you should be using categorical cross-entropy, then you should use a sigmoid output instead.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-10-22T12:11:17.633Z
Answers.Text: Thanks! changing the loss and adding batch normalization and data augmentation helped in getting the Validation loss down to 55 and val_accuracy to 80. But it remains stuck at those values even beyond 30 epochs.
Answers.UserDisplayName: Rishhh
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-10-19T12:48:38.990Z
Answers.LastActivityDate: 2024-10-19T12:48:38.990Z
Answers.UserDisplayName: Gabriel Ribeiro C.
Answers.Body: <p>You can try <a href="https://en.wikipedia.org/wiki/Batch_normalization" rel="nofollow noreferrer">Batch Normalization</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-10-17T12:15:54.940Z
Answers.LastActivityDate: 2024-10-17T12:15:54.940Z
Answers.UserDisplayName: vishal
Body: <p>I'm using 3000+ retinopathy images in my CNN model. The accuracy remains around 77 to 80, how do i improve the accuracy value and reduce loss value?</p>
  <p>I've tried dropout and Adam optimizer to increase it to 80 from around 76. Adding more layers seems to overfit the data.</p>
  <pre><code>    datagen = ImageDataGenerator(rescale=1./255, validation_split=0.2)
      
      train_gen = datagen.flow_from_directory(
          directory,
          target_size=(150, 150),
          batch_size=32,
          class_mode='categorical',
          subset='training'
      )
  
      val_gen = datagen.flow_from_directory(
          directory,
          target_size=(150, 150),
          batch_size=32,
          class_mode='categorical',
          subset='validation'
      )
      
      return train_gen, val_gen
  
  # Build the CNN model
  model = Sequential([
      Conv2D(32, (3, 3), activation='relu', input_shape=(150, 150, 3)),
      MaxPooling2D(pool_size=(2, 2)),
      Conv2D(64, (3, 3), activation='relu'),
      MaxPooling2D(pool_size=(2, 2)),
      Conv2D(128, (3, 3), activation='relu'),
      MaxPooling2D(pool_size=(2, 2)),
      Conv2D(256, (3, 3), activation='relu'),
      MaxPooling2D(pool_size=(2, 2)),
      Conv2D(512, (3, 3), activation='relu'),
      MaxPooling2D(pool_size=(2, 2)),
      Flatten(),
      Dense(512, activation='relu',kernel_regularizer=l2(0.01)),
      Dropout(0.5),
      Dense(5, activation='softmax')
  ])
  type her# Compile the model
  model.compile(optimizer=Adam(learning_rate=0.001),
                loss='categorical_crossentropy',
                metrics=['accuracy'])
  
  # Callbacks
  early_stopping = EarlyStopping(monitor='val_loss', patience = 8)
  model_checkpoint = ModelCheckpoint('best_model.keras', save_best_only=True, monitor='val_loss')
  
  # Train the model
  history = model.fit(
      train_gen,
      epochs= 30,
      shuffle=True,  
      validation_data=val_gen,
      callbacks=[early_stopping, model_checkpoint]e```
  
  ```Epoch 3/30
  92/92 ━━━━━━━━━━━━━━━━━━━━ 35s 371ms/step - accuracy: 0.6982 - loss: 0.8874 - val_accuracy: 0.7291 - val_loss: 0.8297
  Epoch 4/30
  92/92 ━━━━━━━━━━━━━━━━━━━━ 36s 383ms/step - accuracy: 0.7297 - loss: 0.7982 - val_accuracy: 0.6936 - val_loss: 0.8326
  Epoch 5/30
  92/92 ━━━━━━━━━━━━━━━━━━━━ 36s 384ms/step - accuracy: 0.7318 - loss: 0.7914 - val_accuracy: 0.6908 - val_loss: 0.8419
  Epoch 6/30
  7102 - val_accuracy: 0.7510 - val_loss: 0.7556
  Epoch 10/30
  92/92 ━━━━━━━━━━━━━━━━━━━━ 35s 373ms/step - accuracy: 0.7721 - loss: 0.6885 - val_accuracy: 0.7524 - val_loss: 0.7528
  Epoch 11/30
  92/92 ━━━━━━━━━━━━━━━━━━━━ 35s 376ms/step - accuracy: 0.7743 - loss: 0.6734 - val_accuracy: 0.7442 - val_loss: 0.7768
  7102 - val_accuracy: 0.7510 - val_loss: 0.7556
  Epoch 10/30
  92/92 ━━━━━━━━━━━━━━━━━━━━ 35s 373ms/step - accuracy: 0.7721 - loss: 0.6885 - val_accuracy: 0.7524 - val_loss: 0.7528
  Epoch 11/30
  92/92 ━━━━━━━━━━━━━━━━━━━━ 35s 376ms/step - accuracy: 0.7743 - loss: 0.6734 - val_accuracy: 0.7442 - val_loss: 0.7768
  Epoch 12/30
  92/92 ━━━━━━━━━━━━━━━━━━━━ 35s 374ms/step - accuracy: 0.7726 - loss: 0.6500 - val_accuracy: 0.7606 - val_loss: 0.7530
  Epoch 13/30
  92/92 ━━━━━━━━━━━━━━━━━━━━ 35s 378ms/step - accuracy: 0.7776 - loss: 0.6513 - val_accuracy: 0.7538 - val_loss: 0.8011
  Epoch 14/30
  92/92 ━━━━━━━━━━━━━━━━━━━━ 36s 382ms/step - accuracy: 0.7981 - loss: 0.5986 - val_accuracy: 0.7456 - val_loss: 0.7928
  Epoch 15/30
  92/92 ━━━━━━━━━━━━━━━━━━━━ 35s 371ms/step - accuracy: 0.8114 - loss: 0.5714 - val_accuracy: 0.7579 - val_loss: 0.8129
  Epoch 16/30
  92/92 ━━━━━━━━━━━━━━━━━━━━ 36s 384ms/step - accuracy: 0.8061 - loss: 0.5709 - val_accuracy: 0.7524 - val_loss: 0.8745```
  </code></pre>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-10-17T10:04:11.780Z
Comments.Text: Without more context, like simple EDA. It's possible that the data you feed might be garbage that leads the model became GIGO
Comments.UserDisplayName: Muhammad Ikhwan Perwira
Tags: <machine-learning><convolutional-neural-networks><training><python><keras>
Title: How do I improve my model accuracy and val_accuracy for my cnn model?
UserDisplayName: Rishhh
ThreadText: # How do I improve my model accuracy and val_accuracy for my cnn model?
  
  (Asked by: username_0 on 2024-10-17)
  
  I'm using 3000+ retinopathy images in my CNN model. The accuracy remains around 77 to 80, how do i improve the accuracy value and reduce loss value?
  
  I've tried dropout and Adam optimizer to increase it to 80 from around 76. Adding more layers seems to overfit the data.
  
  ```
      datagen = ImageDataGenerator(rescale=1./255, validation_split=0.2)
      
      train_gen = datagen.flow_from_directory(
          directory,
          target_size=(150, 150),
          batch_size=32,
          class_mode='categorical',
          subset='training'
      )
  
      val_gen = datagen.flow_from_directory(
          directory,
          target_size=(150, 150),
          batch_size=32,
          class_mode='categorical',
          subset='validation'
      )
      
      return train_gen, val_gen
  
  # Build the CNN model
  model = Sequential([
      Conv2D(32, (3, 3), activation='relu', input_shape=(150, 150, 3)),
      MaxPooling2D(pool_size=(2, 2)),
      Conv2D(64, (3, 3), activation='relu'),
      MaxPooling2D(pool_size=(2, 2)),
      Conv2D(128, (3, 3), activation='relu'),
      MaxPooling2D(pool_size=(2, 2)),
      Conv2D(256, (3, 3), activation='relu'),
      MaxPooling2D(pool_size=(2, 2)),
      Conv2D(512, (3, 3), activation='relu'),
      MaxPooling2D(pool_size=(2, 2)),
      Flatten(),
      Dense(512, activation='relu',kernel_regularizer=l2(0.01)),
      Dropout(0.5),
      Dense(5, activation='softmax')
  ])
  type her# Compile the model
  model.compile(optimizer=Adam(learning_rate=0.001),
                loss='categorical_crossentropy',
                metrics=['accuracy'])
  
  # Callbacks
  early_stopping = EarlyStopping(monitor='val_loss', patience = 8)
  model_checkpoint = ModelCheckpoint('best_model.keras', save_best_only=True, monitor='val_loss')
  
  # Train the model
  history = model.fit(
      train_gen,
      epochs= 30,
      shuffle=True,  
      validation_data=val_gen,
      callbacks=[early_stopping, model_checkpoint]e```
  
  ```Epoch 3/30
  92/92 ━━━━━━━━━━━━━━━━━━━━ 35s 371ms/step - accuracy: 0.6982 - loss: 0.8874 - val_accuracy: 0.7291 - val_loss: 0.8297
  Epoch 4/30
  92/92 ━━━━━━━━━━━━━━━━━━━━ 36s 383ms/step - accuracy: 0.7297 - loss: 0.7982 - val_accuracy: 0.6936 - val_loss: 0.8326
  Epoch 5/30
  92/92 ━━━━━━━━━━━━━━━━━━━━ 36s 384ms/step - accuracy: 0.7318 - loss: 0.7914 - val_accuracy: 0.6908 - val_loss: 0.8419
  Epoch 6/30
  7102 - val_accuracy: 0.7510 - val_loss: 0.7556
  Epoch 10/30
  92/92 ━━━━━━━━━━━━━━━━━━━━ 35s 373ms/step - accuracy: 0.7721 - loss: 0.6885 - val_accuracy: 0.7524 - val_loss: 0.7528
  Epoch 11/30
  92/92 ━━━━━━━━━━━━━━━━━━━━ 35s 376ms/step - accuracy: 0.7743 - loss: 0.6734 - val_accuracy: 0.7442 - val_loss: 0.7768
  7102 - val_accuracy: 0.7510 - val_loss: 0.7556
  Epoch 10/30
  92/92 ━━━━━━━━━━━━━━━━━━━━ 35s 373ms/step - accuracy: 0.7721 - loss: 0.6885 - val_accuracy: 0.7524 - val_loss: 0.7528
  Epoch 11/30
  92/92 ━━━━━━━━━━━━━━━━━━━━ 35s 376ms/step - accuracy: 0.7743 - loss: 0.6734 - val_accuracy: 0.7442 - val_loss: 0.7768
  Epoch 12/30
  92/92 ━━━━━━━━━━━━━━━━━━━━ 35s 374ms/step - accuracy: 0.7726 - loss: 0.6500 - val_accuracy: 0.7606 - val_loss: 0.7530
  Epoch 13/30
  92/92 ━━━━━━━━━━━━━━━━━━━━ 35s 378ms/step - accuracy: 0.7776 - loss: 0.6513 - val_accuracy: 0.7538 - val_loss: 0.8011
  Epoch 14/30
  92/92 ━━━━━━━━━━━━━━━━━━━━ 36s 382ms/step - accuracy: 0.7981 - loss: 0.5986 - val_accuracy: 0.7456 - val_loss: 0.7928
  Epoch 15/30
  92/92 ━━━━━━━━━━━━━━━━━━━━ 35s 371ms/step - accuracy: 0.8114 - loss: 0.5714 - val_accuracy: 0.7579 - val_loss: 0.8129
  Epoch 16/30
  92/92 ━━━━━━━━━━━━━━━━━━━━ 36s 384ms/step - accuracy: 0.8061 - loss: 0.5709 - val_accuracy: 0.7524 - val_loss: 0.8745```
  
  ```
  
  **Tags:** machine-learning, convolutional-neural-networks, training, python, keras
  
  > **Comments:**
  > **username_1**: Without more context, like simple EDA. It's possible that the data you feed might be garbage that leads the model became GIGO  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2024-10-19. Score: 1
  
  First off, if you really have to use that sequential model of yours then add a [Batch Normalization layer](https://www.tensorflow.org/api_docs/python/tf/keras/layers/BatchNormalization) after each MaxPooling layer. Try getting rid of the kernel regularizer, using a lower dropout rate and GlobalAveragePooling instead of Flatten as well. Otherwise just use a well-known model such as the [EfficientNet](https://github.com/qubvel/efficientnet). Start with the EfficientNetB0 and see if it overfits. If it doesn't, then go up the ladder (B1, B2,...).
  
  Second: use some sort of data augmentation. I recommend using the [Albumentations](https://albumentations.ai/) framework.
  
  Last but not least, you're using the wrong loss. If you have a softmax output, then you should be using sparse cross-entropy instead of categorical. Categorical cross-entropy means that the outputs are simultaneous (i.e. two different labels can happen at the same time), which is the opposite of what a [softmax](https://en.wikipedia.org/wiki/Softmax_function#:%7E:text=The%20softmax%20function%2C%20also%20known,used%20in%20multinomial%20logistic%20regression.) implies (output is a probability distribution, i.e. the sum of the outputs is equal to 1). If you think that the output labels can happen at the same time and that you should be using categorical cross-entropy, then you should use a sigmoid output instead.
  
  > **Comments:**
  > **username_0**: Thanks! changing the loss and adding batch normalization and data augmentation helped in getting the Validation loss down to 55 and val_accuracy to 80. But it remains stuck at those values even beyond 30 epochs.  
  
  ---
  
  ### Answer by username_3 on 2024-10-17. Score: 0
  
  You can try [Batch Normalization](https://en.wikipedia.org/wiki/Batch_normalization).
  
  
  ---
  
  