<p dir="auto">I'm using Keras 2.1.1 and Tensorflow 1.4, Python 3.6, Windows 7.</p>
<p dir="auto">I'm attempting transfer learning using the Inception model.<br>
The code is straight from the Keras Application API, just a few tweaks (using my data).</p>
<p dir="auto">Here is the code</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="from keras.preprocessing import image```
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Model
from keras.layers import Dense, GlobalAveragePooling2D
from keras import backend as K
from keras import optimizers


img_width, img_height = 299, 299
train_data_dir = r'C:\Users\Moondra\Desktop\Keras Applications\data\train'
total_samples = 13581
batch_size = 3
epochs = 5


train_datagen = ImageDataGenerator(
rescale = 1./255,
horizontal_flip = True,
zoom_range = 0.1,
rotation_range=15)



train_generator = train_datagen.flow_from_directory(
train_data_dir,
target_size = (img_height, img_width),
batch_size = batch_size, 
class_mode = 'categorical')  #class_mode = 'categorical'


# create the base pre-trained model
base_model = InceptionV3(weights='imagenet', include_top=False)

# add a global spatial average pooling layer
x = base_model.output
x = GlobalAveragePooling2D()(x)
# let's add a fully-connected layer
x = Dense(1024, activation='relu')(x)
# and a logistic layer -- let's say we have 200 classes
predictions = Dense(12, activation='softmax')(x)

# this is the model we will train
model = Model(input=base_model.input, output=predictions)

# first: train only the top layers (which were randomly initialized)
# i.e. freeze all convolutional InceptionV3 layers
for layer in base_model.layers:
    layer.trainable = False

# compile the model (should be done *after* setting layers to non-trainable)
model.compile(optimizer=optimizers.SGD(lr=0.0001, momentum=0.9), loss='categorical_crossentropy', metrics = ['accuracy'])

# train the model on the new data for a few epochs
model.fit_generator(
train_generator,
steps_per_epoch = 20,
epochs = epochs)


# at this point, the top layers are well trained and we can start fine-tuning
# convolutional layers from inception V3. We will freeze the bottom N layers
# and train the remaining top layers.

# let's visualize layer names and layer indices to see how many layers
# we should freeze:
for i, layer in enumerate(base_model.layers):
   print(i, layer.name)

# we chose to train the top 2 inception blocks, i.e. we will freeze
# the first 172 layers and unfreeze the rest:
for layer in model.layers[:249]:
   layer.trainable = False
for layer in model.layers[249:]:
   layer.trainable = True

# we need to recompile the model for these modifications to take effect
# we use SGD with a low learning rate
from keras.optimizers import SGD
model.compile(optimizer=SGD(lr=0.0001, momentum=0.9), loss='categorical_crossentropy', metrics = ['accuracy'])

# we train our model again (this time fine-tuning the top 2 inception blocks
# alongside the top Dense layers
model.fit_generator(
train_generator,
steps_per_epoch = 25,
epochs = epochs)`


Output is

Found 13581 images belonging to 12 classes.

Warning (from warnings module):
  File &quot;C:\Users\Moondra\Desktop\Keras Applications\keras_transfer_learning_inception_problem_one_epoch.py&quot;, line 44
    model = Model(input=base_model.input, output=predictions)
UserWarning: Update your `Model` call to the Keras 2 API: `Model(inputs=Tensor(&quot;in..., outputs=Tensor(&quot;de...)`
Epoch 1/5

 1/20 [&gt;.............................] - ETA: 38s - loss: 2.8652 - acc: 0.0000e+00����������������������������������������������������������������������������������
 3/20 [===&gt;..........................] - ETA: 12s - loss: 2.6107 - acc: 0.1111    ������������������������������������������������������������������������������
 4/20 [=====&gt;........................] - ETA: 8s - loss: 2.6454 - acc: 0.0833 �����������������������������������������������������������������������������
 5/20 [======&gt;.......................] - ETA: 6s - loss: 2.6483 - acc: 0.0667�����������������������������������������������������������������������������
 6/20 [========&gt;.....................] - ETA: 5s - loss: 2.6863 - acc: 0.0556�����������������������������������������������������������������������������
 7/20 [=========&gt;....................] - ETA: 4s - loss: 2.6230 - acc: 0.0952�����������������������������������������������������������������������������
 8/20 [===========&gt;..................] - ETA: 3s - loss: 2.6212 - acc: 0.0833�����������������������������������������������������������������������������
 9/20 [============&gt;.................] - ETA: 3s - loss: 2.6192 - acc: 0.1111�����������������������������������������������������������������������������
10/20 [==============&gt;...............] - ETA: 2s - loss: 2.6223 - acc: 0.1000�����������������������������������������������������������������������������
11/20 [===============&gt;..............] - ETA: 2s - loss: 2.6626 - acc: 0.0909�����������������������������������������������������������������������������
12/20 [=================&gt;............] - ETA: 2s - loss: 2.6562 - acc: 0.1111�����������������������������������������������������������������������������
13/20 [==================&gt;...........] - ETA: 1s - loss: 2.6436 - acc: 0.1282�����������������������������������������������������������������������������
14/20 [====================&gt;.........] - ETA: 1s - loss: 2.6319 - acc: 0.1190�����������������������������������������������������������������������������
15/20 [=====================&gt;........] - ETA: 1s - loss: 2.6343 - acc: 0.1111
Warning (from warnings module):
  File &quot;C:\Users\Moondra\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\callbacks.py&quot;, line 116
    % delta_t_median)
UserWarning: Method on_batch_end() is slow compared to the batch update (0.102000). Check your callbacks.
�����������������������������������������������������������������������������
16/20 [=======================&gt;......] - ETA: 0s - loss: 2.6310 - acc: 0.1042�����������������������������������������������������������������������������
17/20 [========================&gt;.....] - ETA: 0s - loss: 2.6207 - acc: 0.1176�����������������������������������������������������������������������������
18/20 [==========================&gt;...] - ETA: 0s - loss: 2.6063 - acc: 0.1296�����������������������������������������������������������������������������
19/20 [===========================&gt;..] - ETA: 0s - loss: 2.6056 - acc: 0.1228




It just hangs at the 19/20.

I already asked on stack overflow but no help.

https://stackoverflow.com/questions/47382952/cant-get-past-first-epoch-just-hangs-keras-transfer-learning-inception

"><pre lang="from" class="notranslate"><code class="notranslate">from keras.preprocessing import image```
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Model
from keras.layers import Dense, GlobalAveragePooling2D
from keras import backend as K
from keras import optimizers


img_width, img_height = 299, 299
train_data_dir = r'C:\Users\Moondra\Desktop\Keras Applications\data\train'
total_samples = 13581
batch_size = 3
epochs = 5


train_datagen = ImageDataGenerator(
rescale = 1./255,
horizontal_flip = True,
zoom_range = 0.1,
rotation_range=15)



train_generator = train_datagen.flow_from_directory(
train_data_dir,
target_size = (img_height, img_width),
batch_size = batch_size, 
class_mode = 'categorical')  #class_mode = 'categorical'


# create the base pre-trained model
base_model = InceptionV3(weights='imagenet', include_top=False)

# add a global spatial average pooling layer
x = base_model.output
x = GlobalAveragePooling2D()(x)
# let's add a fully-connected layer
x = Dense(1024, activation='relu')(x)
# and a logistic layer -- let's say we have 200 classes
predictions = Dense(12, activation='softmax')(x)

# this is the model we will train
model = Model(input=base_model.input, output=predictions)

# first: train only the top layers (which were randomly initialized)
# i.e. freeze all convolutional InceptionV3 layers
for layer in base_model.layers:
    layer.trainable = False

# compile the model (should be done *after* setting layers to non-trainable)
model.compile(optimizer=optimizers.SGD(lr=0.0001, momentum=0.9), loss='categorical_crossentropy', metrics = ['accuracy'])

# train the model on the new data for a few epochs
model.fit_generator(
train_generator,
steps_per_epoch = 20,
epochs = epochs)


# at this point, the top layers are well trained and we can start fine-tuning
# convolutional layers from inception V3. We will freeze the bottom N layers
# and train the remaining top layers.

# let's visualize layer names and layer indices to see how many layers
# we should freeze:
for i, layer in enumerate(base_model.layers):
   print(i, layer.name)

# we chose to train the top 2 inception blocks, i.e. we will freeze
# the first 172 layers and unfreeze the rest:
for layer in model.layers[:249]:
   layer.trainable = False
for layer in model.layers[249:]:
   layer.trainable = True

# we need to recompile the model for these modifications to take effect
# we use SGD with a low learning rate
from keras.optimizers import SGD
model.compile(optimizer=SGD(lr=0.0001, momentum=0.9), loss='categorical_crossentropy', metrics = ['accuracy'])

# we train our model again (this time fine-tuning the top 2 inception blocks
# alongside the top Dense layers
model.fit_generator(
train_generator,
steps_per_epoch = 25,
epochs = epochs)`


Output is

Found 13581 images belonging to 12 classes.

Warning (from warnings module):
  File "C:\Users\Moondra\Desktop\Keras Applications\keras_transfer_learning_inception_problem_one_epoch.py", line 44
    model = Model(input=base_model.input, output=predictions)
UserWarning: Update your `Model` call to the Keras 2 API: `Model(inputs=Tensor("in..., outputs=Tensor("de...)`
Epoch 1/5

 1/20 [&gt;.............................] - ETA: 38s - loss: 2.8652 - acc: 0.0000e+00����������������������������������������������������������������������������������
 3/20 [===&gt;..........................] - ETA: 12s - loss: 2.6107 - acc: 0.1111    ������������������������������������������������������������������������������
 4/20 [=====&gt;........................] - ETA: 8s - loss: 2.6454 - acc: 0.0833 �����������������������������������������������������������������������������
 5/20 [======&gt;.......................] - ETA: 6s - loss: 2.6483 - acc: 0.0667�����������������������������������������������������������������������������
 6/20 [========&gt;.....................] - ETA: 5s - loss: 2.6863 - acc: 0.0556�����������������������������������������������������������������������������
 7/20 [=========&gt;....................] - ETA: 4s - loss: 2.6230 - acc: 0.0952�����������������������������������������������������������������������������
 8/20 [===========&gt;..................] - ETA: 3s - loss: 2.6212 - acc: 0.0833�����������������������������������������������������������������������������
 9/20 [============&gt;.................] - ETA: 3s - loss: 2.6192 - acc: 0.1111�����������������������������������������������������������������������������
10/20 [==============&gt;...............] - ETA: 2s - loss: 2.6223 - acc: 0.1000�����������������������������������������������������������������������������
11/20 [===============&gt;..............] - ETA: 2s - loss: 2.6626 - acc: 0.0909�����������������������������������������������������������������������������
12/20 [=================&gt;............] - ETA: 2s - loss: 2.6562 - acc: 0.1111�����������������������������������������������������������������������������
13/20 [==================&gt;...........] - ETA: 1s - loss: 2.6436 - acc: 0.1282�����������������������������������������������������������������������������
14/20 [====================&gt;.........] - ETA: 1s - loss: 2.6319 - acc: 0.1190�����������������������������������������������������������������������������
15/20 [=====================&gt;........] - ETA: 1s - loss: 2.6343 - acc: 0.1111
Warning (from warnings module):
  File "C:\Users\Moondra\AppData\Local\Programs\Python\Python36\lib\site-packages\keras\callbacks.py", line 116
    % delta_t_median)
UserWarning: Method on_batch_end() is slow compared to the batch update (0.102000). Check your callbacks.
�����������������������������������������������������������������������������
16/20 [=======================&gt;......] - ETA: 0s - loss: 2.6310 - acc: 0.1042�����������������������������������������������������������������������������
17/20 [========================&gt;.....] - ETA: 0s - loss: 2.6207 - acc: 0.1176�����������������������������������������������������������������������������
18/20 [==========================&gt;...] - ETA: 0s - loss: 2.6063 - acc: 0.1296�����������������������������������������������������������������������������
19/20 [===========================&gt;..] - ETA: 0s - loss: 2.6056 - acc: 0.1228




It just hangs at the 19/20.

I already asked on stack overflow but no help.

https://stackoverflow.com/questions/47382952/cant-get-past-first-epoch-just-hangs-keras-transfer-learning-inception


</code></pre></div>
<p dir="auto">When <code class="notranslate">model.fit_generator</code> is used with <code class="notranslate">workers=0</code> and subclasses of <a href="https://keras.io/utils/#sequence" rel="nofollow"><code class="notranslate">keras.utils.data_utils.Sequence</code></a> for both training and validation data, API of <code class="notranslate">Sequence</code> is not recognized inside <code class="notranslate">evaluate_generator</code>, it raises:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="  File &quot;.../keras/legacy/interfaces.py&quot;, line 91, in wrapper
    return func(*args, **kwargs)
  File &quot;.../keras/engine/training.py&quot;, line 1415, in fit_generator
    initial_epoch=initial_epoch)
  File &quot;.../keras/engine/training_generator.py&quot;, line 230, in fit_generator
    validation_steps,
  File &quot;.../keras/legacy/interfaces.py&quot;, line 91, in wrapper
    return func(*args, **kwargs)
  File &quot;.../keras/engine/training.py&quot;, line 1469, in evaluate_generator
    verbose=verbose)
  File &quot;.../keras/engine/training_generator.py&quot;, line 298, in evaluate_generator
    else:
ValueError: `steps=None` is only valid for a generator based on the `keras.utils.Sequence` class. Please specify `steps` or use the `keras.utils.Sequence` class."><pre class="notranslate">  <span class="pl-v">File</span> <span class="pl-s">".../keras/legacy/interfaces.py"</span>, <span class="pl-s1">line</span> <span class="pl-c1">91</span>, <span class="pl-s1">in</span> <span class="pl-s1">wrapper</span>
    <span class="pl-k">return</span> <span class="pl-en">func</span>(<span class="pl-c1">*</span><span class="pl-s1">args</span>, <span class="pl-c1">**</span><span class="pl-s1">kwargs</span>)
  <span class="pl-v">File</span> <span class="pl-s">".../keras/engine/training.py"</span>, <span class="pl-s1">line</span> <span class="pl-c1">1415</span>, <span class="pl-s1">in</span> <span class="pl-s1">fit_generator</span>
    <span class="pl-s1">initial_epoch</span><span class="pl-c1">=</span><span class="pl-s1">initial_epoch</span>)
  <span class="pl-v">File</span> <span class="pl-s">".../keras/engine/training_generator.py"</span>, <span class="pl-s1">line</span> <span class="pl-c1">230</span>, <span class="pl-s1">in</span> <span class="pl-s1">fit_generator</span>
    <span class="pl-s1">validation_steps</span>,
  <span class="pl-v">File</span> <span class="pl-s">".../keras/legacy/interfaces.py"</span>, <span class="pl-s1">line</span> <span class="pl-c1">91</span>, <span class="pl-s1">in</span> <span class="pl-s1">wrapper</span>
    <span class="pl-k">return</span> <span class="pl-en">func</span>(<span class="pl-c1">*</span><span class="pl-s1">args</span>, <span class="pl-c1">**</span><span class="pl-s1">kwargs</span>)
  <span class="pl-v">File</span> <span class="pl-s">".../keras/engine/training.py"</span>, <span class="pl-s1">line</span> <span class="pl-c1">1469</span>, <span class="pl-s1">in</span> <span class="pl-s1">evaluate_generator</span>
    <span class="pl-s1">verbose</span><span class="pl-c1">=</span><span class="pl-s1">verbose</span>)
  <span class="pl-v">File</span> <span class="pl-s">".../keras/engine/training_generator.py"</span>, <span class="pl-s1">line</span> <span class="pl-c1">298</span>, <span class="pl-s1">in</span> <span class="pl-s1">evaluate_generator</span>
    <span class="pl-s1">else</span>:
<span class="pl-v">ValueError</span>: <span class="pl-s">`steps=None`</span> <span class="pl-c1">is</span> <span class="pl-s1">only</span> <span class="pl-s1">valid</span> <span class="pl-k">for</span> <span class="pl-s1">a</span> <span class="pl-s1">generator</span> <span class="pl-s1">based</span> <span class="pl-s1">on</span> <span class="pl-s1">the</span> <span class="pl-s">`keras.utils.Sequence`</span> <span class="pl-s1">class</span>. <span class="pl-v">Please</span> <span class="pl-s1">specify</span> <span class="pl-s">`steps`</span> <span class="pl-c1">or</span> <span class="pl-s1">use</span> <span class="pl-s1">the</span> <span class="pl-s">`keras.utils.Sequence`</span> <span class="pl-s1">class</span>.</pre></div>
<p dir="auto">Example code:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="from keras import Sequential
from keras.layers import Dense
from keras.utils.data_utils import Sequence
import numpy as np

class Dataset(Sequence):
    def __getitem__(self, index):
        return np.random.uniform(size=(16, 8)), np.random.uniform(size=(16, 1))
    def __len__(self):
        return 128

model = Sequential([Dense(4, activation='relu', input_shape=(8,)),
                    Dense(1, activation='sigmoid')])
model.compile(loss='mse', optimizer='adam')
model.fit_generator(generator=Dataset(), validation_data=Dataset(),
                    workers=0)"><pre class="notranslate"><span class="pl-k">from</span> <span class="pl-s1">keras</span> <span class="pl-k">import</span> <span class="pl-v">Sequential</span>
<span class="pl-k">from</span> <span class="pl-s1">keras</span>.<span class="pl-s1">layers</span> <span class="pl-k">import</span> <span class="pl-v">Dense</span>
<span class="pl-k">from</span> <span class="pl-s1">keras</span>.<span class="pl-s1">utils</span>.<span class="pl-s1">data_utils</span> <span class="pl-k">import</span> <span class="pl-v">Sequence</span>
<span class="pl-k">import</span> <span class="pl-s1">numpy</span> <span class="pl-k">as</span> <span class="pl-s1">np</span>

<span class="pl-k">class</span> <span class="pl-v">Dataset</span>(<span class="pl-v">Sequence</span>):
    <span class="pl-k">def</span> <span class="pl-en">__getitem__</span>(<span class="pl-s1">self</span>, <span class="pl-s1">index</span>):
        <span class="pl-k">return</span> <span class="pl-s1">np</span>.<span class="pl-s1">random</span>.<span class="pl-en">uniform</span>(<span class="pl-s1">size</span><span class="pl-c1">=</span>(<span class="pl-c1">16</span>, <span class="pl-c1">8</span>)), <span class="pl-s1">np</span>.<span class="pl-s1">random</span>.<span class="pl-en">uniform</span>(<span class="pl-s1">size</span><span class="pl-c1">=</span>(<span class="pl-c1">16</span>, <span class="pl-c1">1</span>))
    <span class="pl-k">def</span> <span class="pl-en">__len__</span>(<span class="pl-s1">self</span>):
        <span class="pl-k">return</span> <span class="pl-c1">128</span>

<span class="pl-s1">model</span> <span class="pl-c1">=</span> <span class="pl-v">Sequential</span>([<span class="pl-v">Dense</span>(<span class="pl-c1">4</span>, <span class="pl-s1">activation</span><span class="pl-c1">=</span><span class="pl-s">'relu'</span>, <span class="pl-s1">input_shape</span><span class="pl-c1">=</span>(<span class="pl-c1">8</span>,)),
                    <span class="pl-v">Dense</span>(<span class="pl-c1">1</span>, <span class="pl-s1">activation</span><span class="pl-c1">=</span><span class="pl-s">'sigmoid'</span>)])
<span class="pl-s1">model</span>.<span class="pl-en">compile</span>(<span class="pl-s1">loss</span><span class="pl-c1">=</span><span class="pl-s">'mse'</span>, <span class="pl-s1">optimizer</span><span class="pl-c1">=</span><span class="pl-s">'adam'</span>)
<span class="pl-s1">model</span>.<span class="pl-en">fit_generator</span>(<span class="pl-s1">generator</span><span class="pl-c1">=</span><span class="pl-v">Dataset</span>(), <span class="pl-s1">validation_data</span><span class="pl-c1">=</span><span class="pl-v">Dataset</span>(),
                    <span class="pl-s1">workers</span><span class="pl-c1">=</span><span class="pl-c1">0</span>)</pre></div>
<p dir="auto">Issue can be fixed <a href="https://github.com/keras-team/keras/blob/7205d903fbc079bb99fbae0e3c02e6d2b4d227f0/keras/engine/training_generator.py#L124">here</a> by replacing:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="if isinstance(val_data, Sequence):
    val_enqueuer_gen = iter(val_data)"><pre class="notranslate"><span class="pl-k">if</span> <span class="pl-en">isinstance</span>(<span class="pl-s1">val_data</span>, <span class="pl-v">Sequence</span>):
    <span class="pl-s1">val_enqueuer_gen</span> <span class="pl-c1">=</span> <span class="pl-en">iter</span>(<span class="pl-s1">val_data</span>)</pre></div>
<p dir="auto">with</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="if isinstance(val_data, Sequence):
    val_enqueuer_gen = iter(val_data)
    validation_steps = len(val_data)"><pre class="notranslate"><span class="pl-k">if</span> <span class="pl-en">isinstance</span>(<span class="pl-s1">val_data</span>, <span class="pl-v">Sequence</span>):
    <span class="pl-s1">val_enqueuer_gen</span> <span class="pl-c1">=</span> <span class="pl-en">iter</span>(<span class="pl-s1">val_data</span>)
    <span class="pl-s1">validation_steps</span> <span class="pl-c1">=</span> <span class="pl-en">len</span>(<span class="pl-s1">val_data</span>)</pre></div>