<p dir="auto">Hi, I'm trying to build a model with functional API. But I got some error only when I use BatchNorm layer . A toy example:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="import numpy as np
from keras.layers import Input, BatchNormalization, Dense
from keras.models import Model

# model1:
input1 = Input((10,))
bn1 = BatchNormalization()(input1)
out1 = Dense(2, activation='softmax')(bn1)
model1 = Model(input1, out1)

# model2:
input2 = Input((10,))
out2 = Dense(10, activation='relu')(input2)
model2 = Model(input2, out2)

# model3 is just a simple stack of model1 and model2:
input3 = Input((10,))
out3 = model1(model2(input3))
model3 = Model(input3, out3)

# compile and train:
model1.compile(loss='categorical_crossentropy',
               optimizer='adam',
               metrics=['accuracy'])
y = np.zeros((10, 2))
x = np.zeros((10, 10))
model1.train_on_batch(x, y)"><pre class="notranslate"><code class="notranslate">import numpy as np
from keras.layers import Input, BatchNormalization, Dense
from keras.models import Model

# model1:
input1 = Input((10,))
bn1 = BatchNormalization()(input1)
out1 = Dense(2, activation='softmax')(bn1)
model1 = Model(input1, out1)

# model2:
input2 = Input((10,))
out2 = Dense(10, activation='relu')(input2)
model2 = Model(input2, out2)

# model3 is just a simple stack of model1 and model2:
input3 = Input((10,))
out3 = model1(model2(input3))
model3 = Model(input3, out3)

# compile and train:
model1.compile(loss='categorical_crossentropy',
               optimizer='adam',
               metrics=['accuracy'])
y = np.zeros((10, 2))
x = np.zeros((10, 10))
model1.train_on_batch(x, y)
</code></pre></div>
<p dir="auto">Then I get  <code class="notranslate">InvalidArgumentError: You must feed a value for placeholder tensor 'input_3' with dtype float</code>. But model1 has nothing to do with input3 if I train it separately. The weird thing is that if I remove bn1 of model1 or change it to layers like Dense, this error will disappear.</p>
<p dir="auto">I use tensorflow backend and both tensorflow and keras are updated. Do I use functional API in the wrong way or it is a potential bug?</p>
<p dir="auto">The problem I am facing is that the input size is variable and I don't know how to get a symbolic shape in the middle of the graph.</p>
<p dir="auto">Is there a way to:</p>
<ol dir="auto">
<li>Get the <em>symbolic</em> shape of a layer</li>
<li>Specify a <em>symbolic</em> (i.e. variable at run time) <code class="notranslate">target_shape</code> in <code class="notranslate">Reshape(target_shape)</code></li>
</ol>
<p dir="auto">At the moment I came up with this ugly workaround. Is there something better?</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="x_shape = [10, None, None, None, 3]   # batch + 4 axes
model = models.Sequential() 
model.add(Convolution2D(64, 3, 3, border_mode='same', input_shape=x_shape[1:]))
# Will this cause the theano graph to be duplicated??
sh = model.get_output().shape
# Is there an easier way with core.Reshape??
model.add(Lambda(lambda x: x.flatten(ndim=2), output_shape=flattened_shape))
model.add(Activation(K.softmax))
reshape = Lambda(lambda x: x.reshape((sh[0], sh[1], T.prod(sh[2:]))))

# this is what I'd like to do instead
# model.add(Reshape((x.shape[0], x.shape[1], x.shape[2]*x.shape[3])))"><pre class="notranslate"><span class="pl-s1">x_shape</span> <span class="pl-c1">=</span> [<span class="pl-c1">10</span>, <span class="pl-c1">None</span>, <span class="pl-c1">None</span>, <span class="pl-c1">None</span>, <span class="pl-c1">3</span>]   <span class="pl-c"># batch + 4 axes</span>
<span class="pl-s1">model</span> <span class="pl-c1">=</span> <span class="pl-s1">models</span>.<span class="pl-v">Sequential</span>() 
<span class="pl-s1">model</span>.<span class="pl-en">add</span>(<span class="pl-v">Convolution2D</span>(<span class="pl-c1">64</span>, <span class="pl-c1">3</span>, <span class="pl-c1">3</span>, <span class="pl-s1">border_mode</span><span class="pl-c1">=</span><span class="pl-s">'same'</span>, <span class="pl-s1">input_shape</span><span class="pl-c1">=</span><span class="pl-s1">x_shape</span>[<span class="pl-c1">1</span>:]))
<span class="pl-c"># Will this cause the theano graph to be duplicated??</span>
<span class="pl-s1">sh</span> <span class="pl-c1">=</span> <span class="pl-s1">model</span>.<span class="pl-en">get_output</span>().<span class="pl-s1">shape</span>
<span class="pl-c"># Is there an easier way with core.Reshape??</span>
<span class="pl-s1">model</span>.<span class="pl-en">add</span>(<span class="pl-v">Lambda</span>(<span class="pl-k">lambda</span> <span class="pl-s1">x</span>: <span class="pl-s1">x</span>.<span class="pl-en">flatten</span>(<span class="pl-s1">ndim</span><span class="pl-c1">=</span><span class="pl-c1">2</span>), <span class="pl-s1">output_shape</span><span class="pl-c1">=</span><span class="pl-s1">flattened_shape</span>))
<span class="pl-s1">model</span>.<span class="pl-en">add</span>(<span class="pl-v">Activation</span>(<span class="pl-v">K</span>.<span class="pl-s1">softmax</span>))
<span class="pl-s1">reshape</span> <span class="pl-c1">=</span> <span class="pl-v">Lambda</span>(<span class="pl-k">lambda</span> <span class="pl-s1">x</span>: <span class="pl-s1">x</span>.<span class="pl-en">reshape</span>((<span class="pl-s1">sh</span>[<span class="pl-c1">0</span>], <span class="pl-s1">sh</span>[<span class="pl-c1">1</span>], <span class="pl-v">T</span>.<span class="pl-en">prod</span>(<span class="pl-s1">sh</span>[<span class="pl-c1">2</span>:]))))

<span class="pl-c"># this is what I'd like to do instead</span>
<span class="pl-c"># model.add(Reshape((x.shape[0], x.shape[1], x.shape[2]*x.shape[3])))</span></pre></div>