<p dir="auto">This bug is easy to reproduce when training GANs but probably occurs in other cases as well. In GANs, batch normalization cannot be used in the discriminator. If BN is used in discriminator you will get errors like:</p>
<div class="highlight highlight-source-shell notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="W tensorflow/core/framework/op_kernel.cc:993] Invalid argument: You must feed a value for placeholder tensor 'input_1' with dtype float
         [[Node: input_1 = Placeholder[dtype=DT_FLOAT, shape=[], _device=&quot;/job:localhost/replica:0/task:0/cpu:0&quot;]()]]
Traceback (most recent call last):
  File &quot;/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/client/session.py&quot;, line 1022, in _do_call
    return fn(*args)
  File &quot;/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/client/session.py&quot;, line 1004, in _run_fn
    status, run_metadata)
  File &quot;/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/contextlib.py&quot;, line 66, in __exit__
    next(self.gen)
  File &quot;/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/framework/errors_impl.py&quot;, line 469, in raise_exception_on_not_ok_status
    pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'input_1' with dtype float
         [[Node: input_1 = Placeholder[dtype=DT_FLOAT, shape=[], _device=&quot;/job:localhost/replica:0/task:0/cpu:0&quot;]()]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File &quot;gan.py&quot;, line 329, in &lt;module&gt;
    main(sys.argv[1], gen_model_path, disc_model_path)
  File &quot;gan.py&quot;, line 321, in main
    adversarial_training(data_dir, generator_model_path, discriminator_model_path)
  File &quot;gan.py&quot;, line 262, in adversarial_training
    loss_real, loss_generated = train_discriminator_step()
  File &quot;gan.py&quot;, line 246, in train_discriminator_step
    l_r = discriminator_model.train_on_batch(x, -np.ones(batch_size))
  File &quot;/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/keras/engine/training.py&quot;, line 1610, in train_on_batch
    outputs = self.train_function(ins)
  File &quot;/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py&quot;, line 2065, in __call__
    feed_dict=feed_dict)
  File &quot;/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/client/session.py&quot;, line 767, in run
    run_metadata_ptr)
  File &quot;/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/client/session.py&quot;, line 965, in _run
    feed_dict_string, options, run_metadata)
  File &quot;/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/client/session.py&quot;, line 1015, in _do_run
    target_list, options, run_metadata)
  File &quot;/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/client/session.py&quot;, line 1035, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'input_1' with dtype float
         [[Node: input_1 = Placeholder[dtype=DT_FLOAT, shape=[], _device=&quot;/job:localhost/replica:0/task:0/cpu:0&quot;]()]]

Caused by op 'input_1', defined at:
  File &quot;gan.py&quot;, line 329, in &lt;module&gt;
    main(sys.argv[1], gen_model_path, disc_model_path)
  File &quot;gan.py&quot;, line 321, in main
    adversarial_training(data_dir, generator_model_path, discriminator_model_path)
  File &quot;gan.py&quot;, line 155, in adversarial_training
    generator_input_tensor = layers.Input(shape=(rand_dim, ))
  File &quot;/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/keras/engine/topology.py&quot;, line 1373, in Input
    input_tensor=tensor)
  File &quot;/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/keras/engine/topology.py&quot;, line 1284, in __init__
    name=self.name)
  File &quot;/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py&quot;, line 346, in placeholder
    x = tf.placeholder(dtype, shape=shape, name=name)
  File &quot;/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/ops/array_ops.py&quot;, line 1520, in placeholder
    name=name)
  File &quot;/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/ops/gen_array_ops.py&quot;, line 2149, in _placeholder
    name=name)
  File &quot;/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py&quot;, line 763, in apply_op
    op_def=op_def)
  File &quot;/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/framework/ops.py&quot;, line 2395, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File &quot;/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/framework/ops.py&quot;, line 1264, in __init__
    self._traceback = _extract_stack()

InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'input_1' with dtype float
         [[Node: input_1 = Placeholder[dtype=DT_FLOAT, shape=[], _device=&quot;/job:localhost/replica:0/task:0/cpu:0&quot;]()]]"><pre class="notranslate">W tensorflow/core/framework/op_kernel.cc:993] Invalid argument: You must feed a value <span class="pl-k">for</span> placeholder tensor <span class="pl-s"><span class="pl-pds">'</span>input_1<span class="pl-pds">'</span></span> with dtype float
         [[Node: input_1 <span class="pl-k">=</span> Placeholder[dtype<span class="pl-k">=</span>DT_FLOAT, shape<span class="pl-k">=</span>[], _device<span class="pl-k">=</span><span class="pl-s"><span class="pl-pds">"</span>/job:localhost/replica:0/task:0/cpu:0<span class="pl-pds">"</span></span>]()]]
Traceback (most recent call last):
  File <span class="pl-s"><span class="pl-pds">"</span>/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/client/session.py<span class="pl-pds">"</span></span>, line 1022, <span class="pl-k">in</span> _do_call
    <span class="pl-k">return</span> fn(<span class="pl-k">*</span>args)
  File <span class="pl-s"><span class="pl-pds">"</span>/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/client/session.py<span class="pl-pds">"</span></span>, line 1004, <span class="pl-k">in</span> _run_fn
    status, run_metadata)
  File <span class="pl-s"><span class="pl-pds">"</span>/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/contextlib.py<span class="pl-pds">"</span></span>, line 66, <span class="pl-k">in</span> __exit__
    next(self.gen)
  File <span class="pl-s"><span class="pl-pds">"</span>/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/framework/errors_impl.py<span class="pl-pds">"</span></span>, line 469, <span class="pl-k">in</span> raise_exception_on_not_ok_status
    pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value <span class="pl-k">for</span> placeholder tensor <span class="pl-s"><span class="pl-pds">'</span>input_1<span class="pl-pds">'</span></span> with dtype float
         [[Node: input_1 <span class="pl-k">=</span> Placeholder[dtype<span class="pl-k">=</span>DT_FLOAT, shape<span class="pl-k">=</span>[], _device<span class="pl-k">=</span><span class="pl-s"><span class="pl-pds">"</span>/job:localhost/replica:0/task:0/cpu:0<span class="pl-pds">"</span></span>]()]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File <span class="pl-s"><span class="pl-pds">"</span>gan.py<span class="pl-pds">"</span></span>, line 329, <span class="pl-k">in</span> <span class="pl-k">&lt;</span>module<span class="pl-k">&gt;</span>
    main(sys.argv[1], gen_model_path, disc_model_path)
  File <span class="pl-s"><span class="pl-pds">"</span>gan.py<span class="pl-pds">"</span></span>, line 321, <span class="pl-k">in</span> main
    adversarial_training(data_dir, generator_model_path, discriminator_model_path)
  File <span class="pl-s"><span class="pl-pds">"</span>gan.py<span class="pl-pds">"</span></span>, line 262, <span class="pl-k">in</span> adversarial_training
    loss_real, loss_generated = <span class="pl-en">train_discriminator_step</span>()
  File <span class="pl-s"><span class="pl-pds">"</span>gan.py<span class="pl-pds">"</span></span>, line 246, <span class="pl-k">in</span> train_discriminator_step
    l_r = discriminator_model.train_on_batch(x, -np.ones(batch_size))
  File <span class="pl-s"><span class="pl-pds">"</span>/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/keras/engine/training.py<span class="pl-pds">"</span></span>, line 1610, <span class="pl-k">in</span> train_on_batch
    outputs = self.train_function(ins)
  File <span class="pl-s"><span class="pl-pds">"</span>/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py<span class="pl-pds">"</span></span>, line 2065, <span class="pl-k">in</span> __call__
    feed_dict=feed_dict)
  File <span class="pl-s"><span class="pl-pds">"</span>/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/client/session.py<span class="pl-pds">"</span></span>, line 767, <span class="pl-k">in</span> run
    run_metadata_ptr)
  File <span class="pl-s"><span class="pl-pds">"</span>/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/client/session.py<span class="pl-pds">"</span></span>, line 965, <span class="pl-k">in</span> _run
    feed_dict_string, options, run_metadata)
  File <span class="pl-s"><span class="pl-pds">"</span>/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/client/session.py<span class="pl-pds">"</span></span>, line 1015, <span class="pl-k">in</span> _do_run
    target_list, options, run_metadata)
  File <span class="pl-s"><span class="pl-pds">"</span>/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/client/session.py<span class="pl-pds">"</span></span>, line 1035, <span class="pl-k">in</span> _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value <span class="pl-k">for</span> placeholder tensor <span class="pl-s"><span class="pl-pds">'</span>input_1<span class="pl-pds">'</span></span> with dtype float
         [[Node: input_1 <span class="pl-k">=</span> Placeholder[dtype<span class="pl-k">=</span>DT_FLOAT, shape<span class="pl-k">=</span>[], _device<span class="pl-k">=</span><span class="pl-s"><span class="pl-pds">"</span>/job:localhost/replica:0/task:0/cpu:0<span class="pl-pds">"</span></span>]()]]

Caused by op <span class="pl-s"><span class="pl-pds">'</span>input_1<span class="pl-pds">'</span></span>, defined at:
  File <span class="pl-s"><span class="pl-pds">"</span>gan.py<span class="pl-pds">"</span></span>, line 329, <span class="pl-k">in</span> <span class="pl-k">&lt;</span>module<span class="pl-k">&gt;</span>
    main(sys.argv[1], gen_model_path, disc_model_path)
  File <span class="pl-s"><span class="pl-pds">"</span>gan.py<span class="pl-pds">"</span></span>, line 321, <span class="pl-k">in</span> main
    adversarial_training(data_dir, generator_model_path, discriminator_model_path)
  File <span class="pl-s"><span class="pl-pds">"</span>gan.py<span class="pl-pds">"</span></span>, line 155, <span class="pl-k">in</span> adversarial_training
    generator_input_tensor = layers.Input(shape=(rand_dim, ))
  File <span class="pl-s"><span class="pl-pds">"</span>/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/keras/engine/topology.py<span class="pl-pds">"</span></span>, line 1373, <span class="pl-k">in</span> Input
    input_tensor=tensor)
  File <span class="pl-s"><span class="pl-pds">"</span>/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/keras/engine/topology.py<span class="pl-pds">"</span></span>, line 1284, <span class="pl-k">in</span> __init__
    name=self.name)
  File <span class="pl-s"><span class="pl-pds">"</span>/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py<span class="pl-pds">"</span></span>, line 346, <span class="pl-k">in</span> placeholder
    x = tf.placeholder(dtype, shape=shape, name=name)
  File <span class="pl-s"><span class="pl-pds">"</span>/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/ops/array_ops.py<span class="pl-pds">"</span></span>, line 1520, <span class="pl-k">in</span> placeholder
    name=name)
  File <span class="pl-s"><span class="pl-pds">"</span>/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/ops/gen_array_ops.py<span class="pl-pds">"</span></span>, line 2149, <span class="pl-k">in</span> _placeholder
    name=name)
  File <span class="pl-s"><span class="pl-pds">"</span>/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py<span class="pl-pds">"</span></span>, line 763, <span class="pl-k">in</span> apply_op
    op_def=op_def)
  File <span class="pl-s"><span class="pl-pds">"</span>/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/framework/ops.py<span class="pl-pds">"</span></span>, line 2395, <span class="pl-k">in</span> create_op
    original_op=self._default_original_op, op_def=op_def)
  File <span class="pl-s"><span class="pl-pds">"</span>/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/tensorflow/python/framework/ops.py<span class="pl-pds">"</span></span>, line 1264, <span class="pl-k">in</span> __init__
    self._traceback = <span class="pl-en">_extract_stack</span>()

InvalidArgumentError (see above <span class="pl-k">for</span> traceback): You must feed a value <span class="pl-k">for</span> placeholder tensor <span class="pl-s"><span class="pl-pds">'</span>input_1<span class="pl-pds">'</span></span> with dtype float
         [[Node: input_1 <span class="pl-k">=</span> Placeholder[dtype<span class="pl-k">=</span>DT_FLOAT, shape<span class="pl-k">=</span>[], _device<span class="pl-k">=</span><span class="pl-s"><span class="pl-pds">"</span>/job:localhost/replica:0/task:0/cpu:0<span class="pl-pds">"</span></span>]()]]</pre></div>
<p dir="auto">What seems to be happening is the computational graphs are separate between <code class="notranslate">generator</code> and <code class="notranslate">discriminator</code> but once you defined <code class="notranslate">combined</code> (something like this):</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="combined_output = discriminator_model(generator_model(generator_input_tensor))
combined_model = models.Model(inputs=[generator_input_tensor], outputs=[combined_output], name='combined')"><pre class="notranslate"><span class="pl-s1">combined_output</span> <span class="pl-c1">=</span> <span class="pl-en">discriminator_model</span>(<span class="pl-en">generator_model</span>(<span class="pl-s1">generator_input_tensor</span>))
<span class="pl-s1">combined_model</span> <span class="pl-c1">=</span> <span class="pl-s1">models</span>.<span class="pl-v">Model</span>(<span class="pl-s1">inputs</span><span class="pl-c1">=</span>[<span class="pl-s1">generator_input_tensor</span>], <span class="pl-s1">outputs</span><span class="pl-c1">=</span>[<span class="pl-s1">combined_output</span>], <span class="pl-s1">name</span><span class="pl-c1">=</span><span class="pl-s">'combined'</span>)</pre></div>
<p dir="auto">things get kinda messed up somehow. basically trying to train a batch on the <code class="notranslate">discriminator</code> will fail b/c now the batch norm layers think a placeholder tensor must be fed for the generators input. although training the generator and combined model will work fine.</p>
<p dir="auto">You can reproduce this in keras 1 and 2 (try <code class="notranslate">cGAN</code> branch for keras 1 and <code class="notranslate">wGAN</code> branch for keras 2) <a href="https://github.com/wayaai/GAN-Sandbox">https://github.com/wayaai/GAN-Sandbox</a></p>
<p dir="auto">In Keras 2 (branch <code class="notranslate">keras-2</code>), the API for pretty much every layer has changed. We will keep supporting the Keras 1 API, and we will do so via <strong>API conversion decorators</strong>.</p>
<p dir="auto">As an example, I have written the conversion decorator for the <code class="notranslate">Dense</code> layer. It was done in this commit: <a class="commit-link" data-hovercard-type="commit" data-hovercard-url="https://github.com/keras-team/keras/commit/5516c8fb42f01622c9a4a31c19cb46ae68cfa0f6/hovercard" href="https://github.com/keras-team/keras/commit/5516c8fb42f01622c9a4a31c19cb46ae68cfa0f6"><tt>5516c8f</tt></a><br>
The decorator code is here: <a href="https://github.com/fchollet/keras/blob/keras-2/keras/legacy/interfaces.py#L15">https://github.com/fchollet/keras/blob/keras-2/keras/legacy/interfaces.py#L15</a><br>
The test for it is here: <a href="https://github.com/fchollet/keras/blob/keras-2/tests/keras/legacy/interface_test.py#L8">https://github.com/fchollet/keras/blob/keras-2/tests/keras/legacy/interface_test.py#L8</a></p>
<p dir="auto">In order to launch Keras 2, we will need such decorators for every layer (and also for training routines and model constructors). If you want to contribute, please add these decorators (in <code class="notranslate">keras/legacy/interfaces.py</code>, with tests in <code class="notranslate">tests/keras/legacy/interface_test.py</code>). You just need to follow the template laid out with the <code class="notranslate">Dense</code> layer decorator.</p>
<p dir="auto">You can leave out the <code class="notranslate">Conv*D</code> layers for now, since they will require special handling due to a positional/keyword argument conflict.</p>
<p dir="auto">Thanks a lot!</p>