Answers.Body: <p>Have you tried increasing the batch size? - try 64, 128 or 256</p>
  <p>Using a very smaller batch size (4 in your case) may not be optimal for the model to converge to the global optima.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-04-06T07:00:02.200Z
Answers.LastActivityDate: 2022-04-06T07:00:02.200Z
Answers.UserDisplayName: Benjamin Cretois
Answers.Body: <p>I don't think this is an overfitting problem, it looks more like the model is not learning anything, but it would be useful to also see the performance on the training data to be sure. By not specifying the learning rate in the optimizer, you are using the default learning rate of 1e-2 which seems quite high. Try passing the optimizer with</p>
  <pre><code>keras.optimizers.SGD(learning_rate=1e-3)
  </code></pre>
  <p>instead for a learning rate of 1e-3 (or maybe try something even lower like 1e-4). And as Benjamin Cretois has mentioned, the batch size is pretty small. If you want to use smaller batch sizes you generally need a smaller learning rate as well. You can also try using other optimizers like Adam, but I think the learning rate is mainly responsible for the bad performance.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-04-08T15:36:36.443Z
Answers.LastActivityDate: 2022-04-08T15:36:36.443Z
Answers.UserDisplayName: Evator
Body: <p>I'm facing the problem of overfitting and I can't deal with it - I tried experimenting with optimizer, but nothing seems appropriate. My model has extremely poor performance on testing data and the loss even rises. Is there anything I missed during the model architecture planning or training?</p>
  <p>I'm working on GTSRB.</p>
  <pre><code>n_epochs = 100
  n_train = 4000
  n_test = 1000
  
  def load_split(basePath, subset_type, n_samples):
      csvPath = basePath + '\\' + subset_type
      #intialize the list of data and labels
      data = []
      labels = []
  
      # load the contents of the CSV file, remove the first line (since it contains the CSV header)
      rows = open(csvPath).read().strip().split(&quot;\n&quot;)[1:n_samples + 1]
      random.shuffle(rows)
  
      #loop over the rows of csv file
      for (i, row) in enumerate(rows):
          #check to see if we should show a status update
          if i &gt; 0 and i % 1000 == 0:
              print(&quot;[INFO] processed {} total images&quot;.format(i))
  
          # split the row into components and then grab the class ID and image path
          (label, imagePath) = row.strip().split(&quot;,&quot;)[-2:]
          # derive the full path to the image file and load it
          imagePath = os.path.sep.join([basePath, imagePath])
          #print(imagePath)
          image = io.imread(imagePath)
  
          #resize the image to be 32x32 pixels, ignoring aspect ratio, and perform CLAHE.
          image = transform.resize(image, (32, 32))
          image = exposure.equalize_adapthist(image, clip_limit = 0.1)
  
          #update the list of data and labels, respectively
          data.append(image)
          labels.append(int(label))
  
      #convert the data and labels into numpy arrays
      data = numpy.array(data)
      labels = numpy.array(labels)
  
      #return a tuple of the data and labels
      return (data, labels)
  
  print(&quot;[INFO] loading training and testing data...&quot;)
  (train_images, train_labels) = load_split(DATASET_PATH, 'Train.csv', n_train)
  (test_images, test_labels) = load_split(DATASET_PATH, 'Test.csv', n_test)
  
  # Normalize pixel values within 0 and 1
  train_images = train_images / 255
  test_images = test_images / 255
  
  train_labels = to_categorical(train_labels, 43)
  test_labels = to_categorical(test_labels, 43)
  
  model = keras.models.Sequential()
  model.add(keras.layers.Flatten(input_shape=[32, 32, 3]))
  model.add(keras.layers.BatchNormalization())
  model.add(keras.layers.Dense(300, activation=&quot;relu&quot;))
  model.add(keras.layers.Dropout(0.4))
  model.add(keras.layers.Dense(300, activation=&quot;relu&quot;))
  model.add(keras.layers.Dropout(0.4))
  model.add(keras.layers.Dense(300, activation=&quot;relu&quot;))
  model.add(keras.layers.Dense(43, activation=&quot;softmax&quot;))
  model.summary()
  
  #Compiling the model
  model.compile(optimizer='sgd', loss='categorical_crossentropy', metrics=['accuracy'])
  
  #Fitting the model
  history= model.fit(train_images,train_labels, epochs=100, batch_size=4,validation_data=(test_images,test_labels))
  </code></pre>
  <p><a href="https://i.sstatic.net/HSJc0.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/HSJc0.png" alt="enter image description here" /></a></p>
  
Tags: <classification><tensorflow><keras>
Title: Overfitting problem - poor performance on test data
UserDisplayName: San
ThreadText: # Overfitting problem - poor performance on test data
  
  (Asked by: username_0 on 2022-04-05)
  
  I'm facing the problem of overfitting and I can't deal with it - I tried experimenting with optimizer, but nothing seems appropriate. My model has extremely poor performance on testing data and the loss even rises. Is there anything I missed during the model architecture planning or training?
  I'm working on GTSRB.
  n_epochs = 100
  n_train = 4000
  n_test = 1000
  
  def load_split(basePath, subset_type, n_samples):
      csvPath = basePath + '\\' + subset_type
      #intialize the list of data and labels
      data = []
      labels = []
  
      # load the contents of the CSV file, remove the first line (since it contains the CSV header)
      rows = open(csvPath).read().strip().split("\n")[1:n_samples + 1]
      random.shuffle(rows)
  
      #loop over the rows of csv file
      for (i, row) in enumerate(rows):
          #check to see if we should show a status update
          if i > 0 and i % 1000 == 0:
              print("[INFO] processed {} total images".format(i))
  
          # split the row into components and then grab the class ID and image path
          (label, imagePath) = row.strip().split(",")[-2:]
          # derive the full path to the image file and load it
          imagePath = os.path.sep.join([basePath, imagePath])
          #print(imagePath)
          image = io.imread(imagePath)
  
          #resize the image to be 32x32 pixels, ignoring aspect ratio, and perform CLAHE.
          image = transform.resize(image, (32, 32))
          image = exposure.equalize_adapthist(image, clip_limit = 0.1)
  
          #update the list of data and labels, respectively
          data.append(image)
          labels.append(int(label))
  
      #convert the data and labels into numpy arrays
      data = numpy.array(data)
      labels = numpy.array(labels)
  
      #return a tuple of the data and labels
      return (data, labels)
  
  print("[INFO] loading training and testing data...")
  (train_images, train_labels) = load_split(DATASET_PATH, 'Train.csv', n_train)
  (test_images, test_labels) = load_split(DATASET_PATH, 'Test.csv', n_test)
  
  # Normalize pixel values within 0 and 1
  train_images = train_images / 255
  test_images = test_images / 255
  
  train_labels = to_categorical(train_labels, 43)
  test_labels = to_categorical(test_labels, 43)
  
  model = keras.models.Sequential()
  model.add(keras.layers.Flatten(input_shape=[32, 32, 3]))
  model.add(keras.layers.BatchNormalization())
  model.add(keras.layers.Dense(300, activation="relu"))
  model.add(keras.layers.Dropout(0.4))
  model.add(keras.layers.Dense(300, activation="relu"))
  model.add(keras.layers.Dropout(0.4))
  model.add(keras.layers.Dense(300, activation="relu"))
  model.add(keras.layers.Dense(43, activation="softmax"))
  model.summary()
  
  #Compiling the model
  model.compile(optimizer='sgd', loss='categorical_crossentropy', metrics=['accuracy'])
  
  #Fitting the model
  history= model.fit(train_images,train_labels, epochs=100, batch_size=4,validation_data=(test_images,test_labels))
  
  **Tags:** classification, tensorflow, keras
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2022-04-06. Score: 2
  
  Have you tried increasing the batch size? - try 64, 128 or 256
  
  Using a very smaller batch size (4 in your case) may not be optimal for the model to converge to the global optima.
  
  
  ---
  
  ### Answer by username_2 on 2022-04-08. Score: 0
  
  I don't think this is an overfitting problem, it looks more like the model is not learning anything, but it would be useful to also see the performance on the training data to be sure. By not specifying the learning rate in the optimizer, you are using the default learning rate of 1e-2 which seems quite high. Try passing the optimizer with
  keras.optimizers.SGD(learning_rate=1e-3)
  
  instead for a learning rate of 1e-3 (or maybe try something even lower like 1e-4). And as Benjamin Cretois has mentioned, the batch size is pretty small. If you want to use smaller batch sizes you generally need a smaller learning rate as well. You can also try using other optimizers like Adam, but I think the learning rate is mainly responsible for the bad performance.
  
  
  ---
  
  