<p>It looks like you're defining an optimizer in the line <code>sgd = optimizers.SGD(lr = 0.01, decay = 1e-6, momentum = 0.9, nesterov = True)</code> and then not using it, since you compile the model with <code>optimizer='adam'</code>. So changing any of the sgd parameters, or even commenting out the line, changes nothing for your model.</p>
<p>you'll need to change to <code>model.compile(optimizer=sgd, loss='categorical_crossentropy', metrics=['accuracy'])</code> to use the <code>sgd</code> optimizer object in the model.</p>

CC BY-SA 4.0
2022-08-18T11:50:09.593Z
Oh,  I didn't pay attention to that. Ok I will try it.
root
CC BY-SA 4.0
2022-08-18T09:26:39.023Z
2022-08-18T09:26:39.023Z
Kroshtan
<p>I am trying to retrain ResNet-50 for iris flower classification in tensorflow (TensorFlow version: 2.3.0) using the following code</p>
<pre><code>import tensorflow as tf
import cv2, random
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
from random import shuffle 
from IPython.display import SVG
import numpy as np # linear algebra
import pandas as pd 
import shutil
import matplotlib.pyplot as plt
%matplotlib inline 
from IPython.display import Image, display
from sklearn.model_selection import train_test_split
import os
print(os.listdir(&quot;./iris recognition/flowers&quot;))

labels = os.listdir(&quot;./iris recognition/flowers&quot;)
num_classes = len(set(labels))
IMAGE_SIZE= 224


# Create model
model = tf.keras.Sequential()
model.add(tf.keras.applications.ResNet50(include_top=False, weights='imagenet'))
model.add(tf.keras.layers.GlobalAveragePooling2D())
model.add(tf.keras.layers.Dropout(0.5))
model.add(tf.keras.layers.Dense(num_classes, activation='softmax'))

# Do not train first layer (ResNet) as it is already pre-trained
model.layers[0].trainable = False

# Compile model
from tensorflow.python.keras import optimizers

sgd = optimizers.SGD(lr = 0.01, decay = 1e-6, momentum = 0.9, nesterov = True)
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
model.summary()
train_folder = './iris recognition/flowers'

image_size = 224
data_generator =  tf.keras.preprocessing.image.ImageDataGenerator(preprocessing_function=tf.keras.applications.resnet50.preprocess_input,
                                    horizontal_flip=True,
                                    width_shift_range=0.2,
                                    height_shift_range=0.2,
                                    validation_split=0.2)# set validation split

train_generator = data_generator.flow_from_directory(
    train_folder,
    target_size=(image_size, image_size),
    batch_size=10,
    class_mode='categorical',
    subset='training'
    )
validation_generator = data_generator.flow_from_directory(
    train_folder,
    target_size=(image_size, image_size),
    batch_size=10,
    class_mode='categorical',
    subset='validation'
    )

NUM_EPOCHS = 70
EARLY_STOP_PATIENCE = 5
from tensorflow.python.keras.callbacks import EarlyStopping, ModelCheckpoint

cb_early_stopper = EarlyStopping(monitor = 'val_loss', patience = EARLY_STOP_PATIENCE)
cb_checkpointer = ModelCheckpoint(filepath = './working/best.hdf5',
                                  monitor = 'val_loss',
                                  save_best_only = True,
                                  mode = 'auto')
import math

fit_history = model.fit(
    train_generator,
    steps_per_epoch=10,
    validation_data=validation_generator,
#     validation_steps=10,
    epochs=NUM_EPOCHS,
    callbacks=[cb_checkpointer, cb_early_stopper])
model.load_weights(&quot;./working/best.hdf5&quot;)
</code></pre>
<p>I tried to change the number of epochs , steps_per_epoch and validation_steps however the model accuracy does not improved as shown in <a href="https://i.sstatic.net/OKz3o.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/OKz3o.png" alt="figure1" /></a> <a href="https://i.sstatic.net/3qmaO.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/3qmaO.png" alt="figure2" /></a></p>
<p>After trying several essais while changing batch size from 32, 64 and 128.For each essai I tried with and without using :</p>
<ul>
<li><p>steps_per_epoch=10.</p>
</li>
<li><p>sgd = optimizers.SGD(lr = 0.01, decay = 1e-6, momentum = 0.9,
nesterov = True)</p>
</li>
</ul>
<p>The best accuracy I achieved is 72.29% in which the training was the most stable which was when using batch size=128 as shwon in <a href="https://i.sstatic.net/oNRL6.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/oNRL6.png" alt="figure3" /></a></p>
<p>Any suggestions please?!</p>

CC BY-SA 4.0
2022-08-15T17:16:55.693Z
Have you tried decreasing the learning rate or increasing the batch size? Your model is fluctuating widely and the lr. or the batch size might be reasons for that.
Mariusmarten
CC BY-SA 4.0
2022-08-15T17:21:19.717Z
I only tried to change the batch size form 24 to 10.
root
CC BY-SA 4.0
2022-08-15T17:23:04.993Z
You could try batch sizes of 32 or 64 (or even 128), this might stabilize the training already.
Mariusmarten
CC BY-SA 4.0
2022-08-16T10:09:42.543Z
Actually, I tried as you said changing with these parameters as follow:  I tried changing the batch size from 32 ,64 and 128 **with sgd** (with and without using steps per epoch) and **without sgd**(with and without using steps per epoch) and the best accuracy I achieved is 72.29%
root
CC BY-SA 4.0
2022-08-16T20:14:59.507Z
Has training become more steady? Have you tried a smaller learning rate as well?
Mariusmarten
CC BY-SA 4.0
2022-08-17T19:29:51.890Z
@Mariusmarten, I have edited the question with the last updates. Considering the learning rate which is in my code **sgd = optimizers.SGD(lr = 0.01, decay = 1e-6, momentum = 0.9, nesterov = True)** I just comment this line and run my code.
root
<tensorflow><residual-networks><iris-dataset>
Retraining ResNet-50 for iris flower classification
root
# Retraining ResNet-50 for iris flower classification

(Asked by: username_0 on 2022-08-15)

I am trying to retrain ResNet-50 for iris flower classification in tensorflow (TensorFlow version: 2.3.0) using the following code
import tensorflow as tf
import cv2, random
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
from random import shuffle 
from IPython.display import SVG
import numpy as np # linear algebra
import pandas as pd 
import shutil
import matplotlib.pyplot as plt
%matplotlib inline 
from IPython.display import Image, display
from sklearn.model_selection import train_test_split
import os
print(os.listdir("./iris recognition/flowers"))

labels = os.listdir("./iris recognition/flowers")
num_classes = len(set(labels))
IMAGE_SIZE= 224


# Create model
model = tf.keras.Sequential()
model.add(tf.keras.applications.ResNet50(include_top=False, weights='imagenet'))
model.add(tf.keras.layers.GlobalAveragePooling2D())
model.add(tf.keras.layers.Dropout(0.5))
model.add(tf.keras.layers.Dense(num_classes, activation='softmax'))

# Do not train first layer (ResNet) as it is already pre-trained
model.layers[0].trainable = False

# Compile model
from tensorflow.python.keras import optimizers

sgd = optimizers.SGD(lr = 0.01, decay = 1e-6, momentum = 0.9, nesterov = True)
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
model.summary()
train_folder = './iris recognition/flowers'

image_size = 224
data_generator =  tf.keras.preprocessing.image.ImageDataGenerator(preprocessing_function=tf.keras.applications.resnet50.preprocess_input,
                                    horizontal_flip=True,
                                    width_shift_range=0.2,
                                    height_shift_range=0.2,
                                    validation_split=0.2)# set validation split

train_generator = data_generator.flow_from_directory(
    train_folder,
    target_size=(image_size, image_size),
    batch_size=10,
    class_mode='categorical',
    subset='training'
    )
validation_generator = data_generator.flow_from_directory(
    train_folder,
    target_size=(image_size, image_size),
    batch_size=10,
    class_mode='categorical',
    subset='validation'
    )

NUM_EPOCHS = 70
EARLY_STOP_PATIENCE = 5
from tensorflow.python.keras.callbacks import EarlyStopping, ModelCheckpoint

cb_early_stopper = EarlyStopping(monitor = 'val_loss', patience = EARLY_STOP_PATIENCE)
cb_checkpointer = ModelCheckpoint(filepath = './working/best.hdf5',
                                  monitor = 'val_loss',
                                  save_best_only = True,
                                  mode = 'auto')
import math

fit_history = model.fit(
    train_generator,
    steps_per_epoch=10,
    validation_data=validation_generator,
#     validation_steps=10,
    epochs=NUM_EPOCHS,
    callbacks=[cb_checkpointer, cb_early_stopper])
model.load_weights("./working/best.hdf5")

I tried to change the number of epochs , steps_per_epoch and validation_steps however the model accuracy does not improved as shown in  
After trying several essais while changing batch size from 32, 64 and 128.For each essai I tried with and without using :

steps_per_epoch=10.

sgd = optimizers.SGD(lr = 0.01, decay = 1e-6, momentum = 0.9,
nesterov = True)


The best accuracy I achieved is 72.29% in which the training was the most stable which was when using batch size=128 as shwon in 
Any suggestions please?!

**Tags:** tensorflow, residual-networks, iris-dataset

> **Comments:**
> **username_1**: Have you tried decreasing the learning rate or increasing the batch size? Your model is fluctuating widely and the lr. or the batch size might be reasons for that.  
> **username_0**: I only tried to change the batch size form 24 to 10.  
> **username_1**: You could try batch sizes of 32 or 64 (or even 128), this might stabilize the training already.  
> **username_0**: Actually, I tried as you said changing with these parameters as follow:  I tried changing the batch size from 32 ,64 and 128 **with sgd** (with and without using steps per epoch) and **without sgd**(with and without using steps per epoch) and the best accuracy I achieved is 72.29%  
> **username_1**: Has training become more steady? Have you tried a smaller learning rate as well?  
> **username_0**: @username_1, I have edited the question with the last updates. Considering the learning rate which is in my code **sgd = optimizers.SGD(lr = 0.01, decay = 1e-6, momentum = 0.9, nesterov = True)** I just comment this line and run my code.  

---

## 1 answers

### Answer by username_2 on 2022-08-18. Score: 1

It looks like you're defining an optimizer in the line `sgd = optimizers.SGD(lr = 0.01, decay = 1e-6, momentum = 0.9, nesterov = True)` and then not using it, since you compile the model with `optimizer='adam'`. So changing any of the sgd parameters, or even commenting out the line, changes nothing for your model.

you'll need to change to `model.compile(optimizer=sgd, loss='categorical_crossentropy', metrics=['accuracy'])` to use the `sgd` optimizer object in the model.

> **Comments:**
> **username_0**: Oh,  I didn't pay attention to that. Ok I will try it.  

---

