<p dir="auto">Tensorflow 1.11.0<br>
Keras 2.2.4<br>
Keras-Preprocessing 1.0.5<br>
Ubuntu 16.04<br>
Python 2.7.12</p>
<p dir="auto">Example code:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="from keras import models, layers
from keras_preprocessing.image import ImageDataGenerator
from keras_applications.imagenet_utils import preprocess_input

train_dir = '/tmp/workspace/Pet_Dataset/train/'
height, width, channels = (128, 128, 3)
batch_size=32

train_datagen = ImageDataGenerator(rescale=1.0/255)
train_generator = train_datagen.flow_from_directory(
              train_dir, target_size=(height, width),
              batch_size=batch_size, class_mode='categorical')

network = models.Sequential()
network.add(layers.Conv2D(32, (3, 3), activation='relu', input_shape=(height, width, channels)))
network.add(layers.MaxPooling2D((2, 2)))
network.add(layers.Conv2D(64, (3, 3), activation='relu'))
network.add(layers.MaxPooling2D((2, 2)))
network.add(layers.Conv2D(128, (3, 3), activation='relu'))
network.add(layers.MaxPooling2D((2, 2)))
network.add(layers.Conv2D(256, (3, 3), activation='relu'))
network.add(layers.GlobalAveragePooling2D())
network.add(layers.Dense(train_generator.num_classes, activation='softmax'))

network.compile(optimizer='RMSProp',
                   loss='categorical_crossentropy',
                   metrics=['accuracy'])
network.summary()

history = network.fit_generator(train_generator, epochs=10) "><pre class="notranslate"><code class="notranslate">from keras import models, layers
from keras_preprocessing.image import ImageDataGenerator
from keras_applications.imagenet_utils import preprocess_input

train_dir = '/tmp/workspace/Pet_Dataset/train/'
height, width, channels = (128, 128, 3)
batch_size=32

train_datagen = ImageDataGenerator(rescale=1.0/255)
train_generator = train_datagen.flow_from_directory(
              train_dir, target_size=(height, width),
              batch_size=batch_size, class_mode='categorical')

network = models.Sequential()
network.add(layers.Conv2D(32, (3, 3), activation='relu', input_shape=(height, width, channels)))
network.add(layers.MaxPooling2D((2, 2)))
network.add(layers.Conv2D(64, (3, 3), activation='relu'))
network.add(layers.MaxPooling2D((2, 2)))
network.add(layers.Conv2D(128, (3, 3), activation='relu'))
network.add(layers.MaxPooling2D((2, 2)))
network.add(layers.Conv2D(256, (3, 3), activation='relu'))
network.add(layers.GlobalAveragePooling2D())
network.add(layers.Dense(train_generator.num_classes, activation='softmax'))

network.compile(optimizer='RMSProp',
                   loss='categorical_crossentropy',
                   metrics=['accuracy'])
network.summary()

history = network.fit_generator(train_generator, epochs=10) 
</code></pre></div>
<p dir="auto">breaks with Tensorflow 1.11.0 with <code class="notranslate">ValueError: steps_per_epoch=None is only valid for a generator based on the keras.utils.Sequence class. Please specify steps_per_epoch or use the keras.utils.Sequence class.</code>.</p>
<p dir="auto">Tensorflow 1.10.0 and below works.</p>
<p dir="auto">Furthermore, providing steps_per_epoch and applying multiprocessing with workers &gt; 1:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="history = network.fit_generator(train_generator, epochs=10, 
    steps_per_epoch=train_generator.n/batch_size,
    use_multiprocessing=True, workers=2)"><pre class="notranslate"><code class="notranslate">history = network.fit_generator(train_generator, epochs=10, 
    steps_per_epoch=train_generator.n/batch_size,
    use_multiprocessing=True, workers=2)
</code></pre></div>
<p dir="auto">causes a UserWarning about multiple workers duplicate data.</p>
<p dir="auto">I think this is caused by wrong inheritance of ImageDataGenerator.</p>
<p dir="auto">When printing the exact class that is used in <code class="notranslate">keras_preprocessing/image.py</code>, I get <code class="notranslate">&lt;class 'tensorflow.python.keras.utils.data_utils.Sequence'&gt;</code>.</p>
<p dir="auto">But <code class="notranslate">fit_generator()</code> checks if the generator is a subclass of <code class="notranslate">&lt;class 'keras.utils.data_utils.Sequence'&gt;</code>, so it seems to assume that the ImageDataGenerator instance is not of type Sequence, causing above problems.</p>
<p dir="auto">I notice there is a difference in how images are pre-processed in the <a href="https://github.com/fchollet/keras/blob/master/examples/neural_style_transfer.py">neural_style_transfer example</a> and the <a href="https://gist.github.com/baraldilorenzo/07d7802847aaad0a35d3">recommended way</a> of using VGG16 model. Specially,</p>
<ul dir="auto">
<li>The color channels in neural_style_transfer example is RGB (as read by scipy.misc.imread), whereas the order in original VGG is expected to be BGR.</li>
<li>The mean-centering step is not performed in the example, i.e.,</li>
</ul>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="im[:,:,0] -= 103.939
im[:,:,1] -= 116.779
im[:,:,2] -= 123.68"><pre class="notranslate"><code class="notranslate">im[:,:,0] -= 103.939
im[:,:,1] -= 116.779
im[:,:,2] -= 123.68
</code></pre></div>
<p dir="auto">Some of my experiments showed that these differences could result in sub-optimal results in image classification, e.g., <a href="https://www.google.com.sg/?ion=1&amp;espv=2#q=red%20fox" rel="nofollow">red fox</a> misclassified to <a href="https://www.google.com.sg/?ion=1&amp;espv=2#q=grey%20fox" rel="nofollow">grey fox</a> due to the wrong order of color channel.</p>
<p dir="auto">However, when it comes to generating "artistic" images as in the neural_style_transfer example, it seems that the results caused by the difference are not so different.  I can understand there are probably some reasons behind this,</p>
<ul dir="auto">
<li>
<ol dir="auto">
<li>The "content loss" and "total variation loss" defined in the example don't depend on the order of color channels.</li>
</ol>
</li>
<li>
<ol start="2" dir="auto">
<li>The "style loss" might depend on the color orders, because it depends on the difference of "styles"  in the original and the generated image. And this style is measured by the inner-product distribution of different vgg filters (Gram matrix). However, it should be expected that most of vgg filters will be based on a mix of colors (e.g., edges, corners) rather than a single color alone (e.g, blue color for sea).</li>
</ol>
</li>
</ul>
<p dir="auto">Those could be the reasons why I don't see many differences when experimenting with the different color channel encoding in the neural_style_transfer example. <em>But I really want to know whether we can come up with some contrived cases where the different color channel order will result in a difference for the neural_style_transfer example.</em></p>
<p dir="auto">I post this question here because it might lead to a potential bug report to the above example code.</p>