Answers.Body: <p>You have two questions in one.</p>
  <ol>
  <li>Is it maxpool that ruins the model?</li>
  </ol>
  <p>I would say no, the maxpool is a standard operation for convolution networks, it down-samples the intermediate representation to reduce the necessary computations, improve the regularization, and adds translation invariance to some degree. Originally averaging was used to downsample over few neighbor pixels, for example, 2x2 were averaged to one pixel. Then it was discovered max-pool often performs better in practice, where you took the max value out of these 2x2 pixels. The way you applied is ok in general.</p>
  <ol start="2">
  <li>Why the accuracy is not that great?</li>
  </ol>
  <p>I see two issues here - first one is COCO dataset is not a classification dataset. It's an <em>object detection</em> dataset and there are many objects on the same image. I.e. there is an image with a person on a bicycle and a car behind him. Which class the model should assign - a person, a bicycle, or a car? The model can't know. To check if it's the issue try <a href="https://www.tensorflow.org/api_docs/python/tf/keras/metrics/TopKCategoricalAccuracy" rel="nofollow noreferrer">top-5 accuracy</a> - it tells if the correct answer would be among top-5 guesses of the network. I would also recommend to watch the images and try to manually guess the class for few dozens of them, that would help to build the intuition</p>
  <p>The second thing is that your model is not that deep and 30% accuracy is not bad, i.e. the random guess would be around 1% and your model doing x30 times better. You could try models like <a href="https://www.tensorflow.org/api_docs/python/tf/keras/applications/ResNet50" rel="nofollow noreferrer">resnet</a> - it's still quite fast, but should be doing noticeably better.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-29T23:25:02.337Z
Answers.Text: That is the exact explanation I was looking for. I truly appreciate it!
Answers.UserDisplayName: DragonflyRobotics
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-29T23:06:37.190Z
Answers.LastActivityDate: 2021-03-29T23:26:57.900Z
Answers.LastEditDate: 2021-03-29T23:26:57.900Z
Answers.UserDisplayName: Kirill Fedyanin
Body: <p>I am trying to make a big classification model using the coco2017 dataset. Here is my code:</p>
  <pre><code>import tensorflow as tf
  from tensorflow import keras
  import numpy as np
  import matplotlib.pyplot as plt
  import IPython.display as display
  from PIL import Image, ImageSequence
  import os
  import pathlib
  from tensorflow.keras.models import Sequential
  from tensorflow.keras.layers import Dense, Conv2D, Flatten, Dropout, MaxPooling2D
  from tensorflow.keras.preprocessing.image import ImageDataGenerator
  import cv2
  import datetime
  
  gpus = tf.config.list_physical_devices('GPU')
  if gpus:
    try:
      # Currently, memory growth needs to be the same across GPUs
      for gpu in gpus:
        tf.config.experimental.set_memory_growth(gpu, True)
      logical_gpus = tf.config.experimental.list_logical_devices('GPU')
      print(len(gpus), &quot;Physical GPUs,&quot;, len(logical_gpus), &quot;Logical GPUs&quot;)
    except RuntimeError as e:
      # Memory growth must be set before GPUs have been initialized
      print(e)
  
  epochs = 100
  steps_per_epoch = 10
  batch_size = 70
  IMG_HEIGHT = 200
  IMG_WIDTH = 200
  
  train_dir = &quot;Train&quot;
  test_dir = &quot;Val&quot;
  
  train_image_generator = ImageDataGenerator(rescale=1. / 255)
  
  test_image_generator = ImageDataGenerator(rescale=1. / 255)
  
  train_data_gen = train_image_generator.flow_from_directory(batch_size=batch_size,
                                                             directory=train_dir,
                                                             shuffle=True,
                                                             target_size=(IMG_HEIGHT, IMG_WIDTH),
                                                             class_mode='sparse')
  
  test_data_gen = test_image_generator.flow_from_directory(batch_size=batch_size,
                                                           directory=test_dir,
                                                           shuffle=True,
                                                           target_size=(IMG_HEIGHT, IMG_WIDTH),
                                                           class_mode='sparse')
  
  model = Sequential([
      Conv2D(265, 3, padding='same', activation='relu', input_shape=(IMG_HEIGHT, IMG_WIDTH ,3)),
      MaxPooling2D(),
      Conv2D(64, 3, padding='same', activation='relu'),
      MaxPooling2D(),
      Conv2D(32, 3, padding='same', activation='relu'),
      MaxPooling2D(),
      Flatten(),
      keras.layers.Dense(256, activation=&quot;relu&quot;),
      keras.layers.Dense(128, activation=&quot;relu&quot;),
      keras.layers.Dense(80, activation=&quot;softmax&quot;)
  ])
  
  optimizer = tf.keras.optimizers.Adam(0.001)
  optimizer.learning_rate.assign(0.0001)
  
  model.compile(optimizer='adam',
                loss=&quot;sparse_categorical_crossentropy&quot;,
                metrics=['accuracy'])
  
  model.summary()
  tf.keras.utils.plot_model(model, to_file=&quot;model.png&quot;, show_shapes=True, show_layer_names=True, rankdir='TB')
  checkpoint_path = &quot;training/cp.ckpt&quot;
  checkpoint_dir = os.path.dirname(checkpoint_path)
  
  cp_callback = tf.keras.callbacks.ModelCheckpoint(filepath=checkpoint_path,
                                                   save_weights_only=True,
                                                   verbose=1)
  
  os.system(&quot;rm -r logs&quot;)
  
  log_dir = &quot;logs/fit/&quot; + datetime.datetime.now().strftime(&quot;%Y%m%d-%H%M%S&quot;)
  tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)
  
  model.load_weights(tf.train.latest_checkpoint(checkpoint_dir))
  history = model.fit(train_data_gen,steps_per_epoch=steps_per_epoch,epochs=epochs,validation_data=test_data_gen,validation_steps=10,callbacks=[cp_callback, tensorboard_callback])
  model.load_weights(tf.train.latest_checkpoint(checkpoint_dir))
  model.save('model.h5', include_optimizer=True)
  
  test_loss, test_acc = model.evaluate(test_data_gen)
  print(&quot;Tested Acc: &quot;, test_acc)
  print(&quot;Tested Acc: &quot;, test_acc*100, &quot;%&quot;)
  </code></pre>
  <p>I have tried different optimizers like <code>SGD</code>, <code>RMSProp</code>, and <code>ADAM</code>. I also tried changing the configuration of the hidden layers. I also tried to change the metrics from <code>accuracy</code> to <code>sparse_categorical_accuracy</code> with no improvement. I cannot go beyond 30% accuracy. My guess is that the <code>MaxPooling</code> is doing something because I just added it but don't know what it means. Can somebody explain what the <code>MaxPooling</code> Layer does and what is stopping my neural network from gaining accuracy?</p>
  
Tags: <convolutional-neural-networks><classification><image-recognition><accuracy><coco-dataset>
Title: Accuracy Not Going Above 30%
UserDisplayName: DragonflyRobotics
ThreadText: # Accuracy Not Going Above 30%
  
  (Asked by: username_0 on 2021-03-29)
  
  I am trying to make a big classification model using the coco2017 dataset. Here is my code:
  
  ```
  import tensorflow as tf
  from tensorflow import keras
  import numpy as np
  import matplotlib.pyplot as plt
  import IPython.display as display
  from PIL import Image, ImageSequence
  import os
  import pathlib
  from tensorflow.keras.models import Sequential
  from tensorflow.keras.layers import Dense, Conv2D, Flatten, Dropout, MaxPooling2D
  from tensorflow.keras.preprocessing.image import ImageDataGenerator
  import cv2
  import datetime
  
  gpus = tf.config.list_physical_devices('GPU')
  if gpus:
    try:
      # Currently, memory growth needs to be the same across GPUs
      for gpu in gpus:
        tf.config.experimental.set_memory_growth(gpu, True)
      logical_gpus = tf.config.experimental.list_logical_devices('GPU')
      print(len(gpus), "Physical GPUs,", len(logical_gpus), "Logical GPUs")
    except RuntimeError as e:
      # Memory growth must be set before GPUs have been initialized
      print(e)
  
  epochs = 100
  steps_per_epoch = 10
  batch_size = 70
  IMG_HEIGHT = 200
  IMG_WIDTH = 200
  
  train_dir = "Train"
  test_dir = "Val"
  
  train_image_generator = ImageDataGenerator(rescale=1. / 255)
  
  test_image_generator = ImageDataGenerator(rescale=1. / 255)
  
  train_data_gen = train_image_generator.flow_from_directory(batch_size=batch_size,
                                                             directory=train_dir,
                                                             shuffle=True,
                                                             target_size=(IMG_HEIGHT, IMG_WIDTH),
                                                             class_mode='sparse')
  
  test_data_gen = test_image_generator.flow_from_directory(batch_size=batch_size,
                                                           directory=test_dir,
                                                           shuffle=True,
                                                           target_size=(IMG_HEIGHT, IMG_WIDTH),
                                                           class_mode='sparse')
  
  model = Sequential([
      Conv2D(265, 3, padding='same', activation='relu', input_shape=(IMG_HEIGHT, IMG_WIDTH ,3)),
      MaxPooling2D(),
      Conv2D(64, 3, padding='same', activation='relu'),
      MaxPooling2D(),
      Conv2D(32, 3, padding='same', activation='relu'),
      MaxPooling2D(),
      Flatten(),
      keras.layers.Dense(256, activation="relu"),
      keras.layers.Dense(128, activation="relu"),
      keras.layers.Dense(80, activation="softmax")
  ])
  
  optimizer = tf.keras.optimizers.Adam(0.001)
  optimizer.learning_rate.assign(0.0001)
  
  model.compile(optimizer='adam',
                loss="sparse_categorical_crossentropy",
                metrics=['accuracy'])
  
  model.summary()
  tf.keras.utils.plot_model(model, to_file="model.png", show_shapes=True, show_layer_names=True, rankdir='TB')
  checkpoint_path = "training/cp.ckpt"
  checkpoint_dir = os.path.dirname(checkpoint_path)
  
  cp_callback = tf.keras.callbacks.ModelCheckpoint(filepath=checkpoint_path,
                                                   save_weights_only=True,
                                                   verbose=1)
  
  os.system("rm -r logs")
  
  log_dir = "logs/fit/" + datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
  tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)
  
  model.load_weights(tf.train.latest_checkpoint(checkpoint_dir))
  history = model.fit(train_data_gen,steps_per_epoch=steps_per_epoch,epochs=epochs,validation_data=test_data_gen,validation_steps=10,callbacks=[cp_callback, tensorboard_callback])
  model.load_weights(tf.train.latest_checkpoint(checkpoint_dir))
  model.save('model.h5', include_optimizer=True)
  
  test_loss, test_acc = model.evaluate(test_data_gen)
  print("Tested Acc: ", test_acc)
  print("Tested Acc: ", test_acc*100, "%")
  
  ```
  
  I have tried different optimizers like `SGD`, `RMSProp`, and `ADAM`. I also tried changing the configuration of the hidden layers. I also tried to change the metrics from `accuracy` to `sparse_categorical_accuracy` with no improvement. I cannot go beyond 30% accuracy. My guess is that the `MaxPooling` is doing something because I just added it but don't know what it means. Can somebody explain what the `MaxPooling` Layer does and what is stopping my neural network from gaining accuracy?
  
  **Tags:** convolutional-neural-networks, classification, image-recognition, accuracy, coco-dataset
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-03-29. Score: 1 | Accepted answer
  
  You have two questions in one.
  
  1. Is it maxpool that ruins the model?
  
  I would say no, the maxpool is a standard operation for convolution networks, it down-samples the intermediate representation to reduce the necessary computations, improve the regularization, and adds translation invariance to some degree. Originally averaging was used to downsample over few neighbor pixels, for example, 2x2 were averaged to one pixel. Then it was discovered max-pool often performs better in practice, where you took the max value out of these 2x2 pixels. The way you applied is ok in general.
  
  2. Why the accuracy is not that great?
  
  I see two issues here - first one is COCO dataset is not a classification dataset. It's an *object detection* dataset and there are many objects on the same image. I.e. there is an image with a person on a bicycle and a car behind him. Which class the model should assign - a person, a bicycle, or a car? The model can't know. To check if it's the issue try [top-5 accuracy](https://www.tensorflow.org/api_docs/python/tf/keras/metrics/TopKCategoricalAccuracy) - it tells if the correct answer would be among top-5 guesses of the network. I would also recommend to watch the images and try to manually guess the class for few dozens of them, that would help to build the intuition
  
  The second thing is that your model is not that deep and 30% accuracy is not bad, i.e. the random guess would be around 1% and your model doing x30 times better. You could try models like [resnet](https://www.tensorflow.org/api_docs/python/tf/keras/applications/ResNet50) - it's still quite fast, but should be doing noticeably better.
  
  > **Comments:**
  > **username_0**: That is the exact explanation I was looking for. I truly appreciate it!  
  
  ---
  
  