<p dir="auto">Please go to Stack Overflow for help and support:</p>
<p dir="auto"><a href="https://stackoverflow.com/questions/tagged/keras" rel="nofollow">https://stackoverflow.com/questions/tagged/keras</a></p>
<p dir="auto">If you open a GitHub issue, here is our policy:</p>
<ol dir="auto">
<li>It must be a bug, a feature request, or a significant problem with the<br>
documentation (for small docs fixes please send a PR instead).</li>
<li>The form below must be filled out.</li>
</ol>
<p dir="auto"><strong>Here's why we have that policy</strong>: Keras developers respond to issues. We want to focus on work that benefits the whole community, e.g., fixing bugs and adding features. Support only helps individuals. GitHub also notifies thousands of people when issues are filed. We want them to see you communicating an interesting problem, rather than being redirected to Stack Overflow.</p>
<hr>
<h3 dir="auto">System information</h3>
<ul dir="auto">
<li><strong>Have I written custom code (as opposed to using a stock example script<br>
provided in Keras)</strong>:</li>
<li><strong>OS Platform and Distribution (e.g., Linux Ubuntu 16.04)</strong>:</li>
<li><strong>TensorFlow installed from (source or binary)</strong>:</li>
<li><strong>TensorFlow version (use command below)</strong>:</li>
<li><strong>Python version</strong>:</li>
<li><strong>Bazel version (if compiling from source)</strong>:</li>
<li><strong>GPU model and memory</strong>:</li>
<li><strong>Exact command to reproduce</strong>:</li>
</ul>
<p dir="auto">You can collect some of this information using our environment capture script:</p>
<p dir="auto"><a href="https://github.com/tensorflow/tensorflow/tree/master/tools/tf_env_collect.sh">https://github.com/tensorflow/tensorflow/tree/master/tools/tf_env_collect.sh</a></p>
<p dir="auto">You can obtain the TensorFlow version with:</p>
<div class="highlight highlight-source-shell notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="python -c &quot;import tensorflow as tf; print(tf.version.GIT_VERSION, tf.version.VERSION)&quot;"><pre class="notranslate">python -c <span class="pl-s"><span class="pl-pds">"</span>import tensorflow as tf; print(tf.version.GIT_VERSION, tf.version.VERSION)<span class="pl-pds">"</span></span></pre></div>
<h3 dir="auto">Describe the problem</h3>
<p dir="auto">Describe the problem clearly here. Be sure to convey here why it's a bug in Keras or why the requested feature is needed.</p>
<h3 dir="auto">Source code / logs</h3>
<p dir="auto">Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached. Try to provide a reproducible test case that is the bare minimum necessary to generate the problem.</p>
<p dir="auto">Currently, Keras builds the computational graph in a <code class="notranslate">top-down</code> fashion. That is, starting from the final output tensor and recursively calling <code class="notranslate">get_output()</code> until reaching the input. This is fine for <code class="notranslate">sequential</code> models, since layers are stacked linearly and the computational flow is a straight line. However, for <code class="notranslate">graph</code> models, the computational flow could have branches, and this <code class="notranslate">top-down</code> fashion will incur redundancy because <code class="notranslate">self.get_output()</code> will "repeat" the downstream computations at the branching point. Consider the following example:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="X = T.vector('X')
B = X * 3
Z1 = B + 1
Z2 = B + 2
Y = Z1 + Z2
theano.printing.pydotprint(Y)"><pre class="notranslate"><code class="notranslate">X = T.vector('X')
B = X * 3
Z1 = B + 1
Z2 = B + 2
Y = Z1 + Z2
theano.printing.pydotprint(Y)
</code></pre></div>
<p dir="auto">The graph looks like <a href="http://i.cs.hku.hk/~pcyin/graph1.png" rel="nofollow">this</a> (as one can imagine)</p>
<p dir="auto">When implemented in Keras:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="# helper layer
class LambdaLayer(Layer):
    def __init__(self, func):
        super(LambdaLayer, self).__init__()
        self.func = func

    def get_output(self, train=False):
        X = self.get_input(train)
        return self.func(X)

g = Graph()
g.add_input('X', ndim=1)

mul = LambdaLayer(lambda x: x * 3)
g.add_node(mul, name='mul', input='X')
add1 = LambdaLayer(lambda x: x + 1)
add2 = LambdaLayer(lambda x: x + 2)
add3 = LambdaLayer(lambda x: x['1'] + x['2'])

g.add_node(add1, name='add1', input='mul')
g.add_node(add2, name='add2', input='mul')
# I modified Keras to support multiple inputs, no Merge layer required here
g.add_node(add3, name='add3', inputs={'1': 'add1', '2': 'add2'}, create_output=True)

theano.printing.pydotprint(g.get_output())"><pre class="notranslate"><code class="notranslate"># helper layer
class LambdaLayer(Layer):
    def __init__(self, func):
        super(LambdaLayer, self).__init__()
        self.func = func

    def get_output(self, train=False):
        X = self.get_input(train)
        return self.func(X)

g = Graph()
g.add_input('X', ndim=1)

mul = LambdaLayer(lambda x: x * 3)
g.add_node(mul, name='mul', input='X')
add1 = LambdaLayer(lambda x: x + 1)
add2 = LambdaLayer(lambda x: x + 2)
add3 = LambdaLayer(lambda x: x['1'] + x['2'])

g.add_node(add1, name='add1', input='mul')
g.add_node(add2, name='add2', input='mul')
# I modified Keras to support multiple inputs, no Merge layer required here
g.add_node(add3, name='add3', inputs={'1': 'add1', '2': 'add2'}, create_output=True)

theano.printing.pydotprint(g.get_output())
</code></pre></div>
<p dir="auto">The graph looks like <a href="http://i.cs.hku.hk/~pcyin/graph2.png" rel="nofollow">this</a>. <code class="notranslate">X * 3</code> repeats twice.</p>
<p dir="auto">Of cause we can count on Theano's optimizer to reduce the redundancy, but when it comes to large graph models with complicated wiring. The optimization will take quite a long time (Actually I have a graph model with lots of branching and it takes 8 hours to compile). Does it seem better to build computational graph in bottom-up fashion?</p>