<p dir="auto"><em>Please make sure that this is a bug. As per our <a href="https://github.com/tensorflow/tensorflow/blob/master/ISSUES.md">GitHub Policy</a>, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:bug_template</em></p>
<p dir="auto"><strong>System information</strong></p>
<ul dir="auto">
<li>Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes</li>
<li>OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Mac</li>
<li>Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: None</li>
<li>TensorFlow installed from (source or binary): binary</li>
<li>TensorFlow version (use command below):</li>
<li>Python version: 2.0 alpha</li>
<li>Bazel version (if compiling from source): None</li>
<li>GCC/Compiler version (if compiling from source): None</li>
<li>CUDA/cuDNN version: None</li>
<li>GPU model and memory: None</li>
</ul>
<p dir="auto">The order of input data is changed.<br>
For example:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="inputs = {'A': [1,2,3,4], 'B':[7,8,9,10]}
A = fc.sequence_numeric_column('A')
B = fc.sequence_numeric_column('B')
columns = [A,B]
seq_feature_layer = keras.experimental.SequenceFeatures(columns)
seq_input, seq_len = seq_feature_layer(inputs)"><pre class="notranslate"><code class="notranslate">inputs = {'A': [1,2,3,4], 'B':[7,8,9,10]}
A = fc.sequence_numeric_column('A')
B = fc.sequence_numeric_column('B')
columns = [A,B]
seq_feature_layer = keras.experimental.SequenceFeatures(columns)
seq_input, seq_len = seq_feature_layer(inputs)
</code></pre></div>
<p dir="auto">And It prints like</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="print(seq_input)
&gt; [[7, 1], [8, 2], [9, 3], [10,4]]"><pre class="notranslate"><code class="notranslate">print(seq_input)
&gt; [[7, 1], [8, 2], [9, 3], [10,4]]
</code></pre></div>
<p dir="auto">This bug seems to exist in all versions of TensorFlow from (at least) <code class="notranslate">1.13</code> to the current <code class="notranslate">2.0</code> nightly under multiple configurations (host os, gpu, python versions...) with CUDA 10.0.</p>
<p dir="auto">It can be reproduced fairly easily:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="with tf.device('gpu:0'):
  v = tf.ones((2**17, 1, 1), dtype=tf.float32)
  print(tf.reduce_sum(tf.matmul(v, v)))"><pre class="notranslate"><span class="pl-k">with</span> <span class="pl-s1">tf</span>.<span class="pl-en">device</span>(<span class="pl-s">'gpu:0'</span>):
  <span class="pl-s1">v</span> <span class="pl-c1">=</span> <span class="pl-s1">tf</span>.<span class="pl-en">ones</span>((<span class="pl-c1">2</span><span class="pl-c1">**</span><span class="pl-c1">17</span>, <span class="pl-c1">1</span>, <span class="pl-c1">1</span>), <span class="pl-s1">dtype</span><span class="pl-c1">=</span><span class="pl-s1">tf</span>.<span class="pl-s1">float32</span>)
  <span class="pl-en">print</span>(<span class="pl-s1">tf</span>.<span class="pl-en">reduce_sum</span>(<span class="pl-s1">tf</span>.<span class="pl-en">matmul</span>(<span class="pl-s1">v</span>, <span class="pl-s1">v</span>)))</pre></div>
<p dir="auto">This prints <code class="notranslate">65535.0</code> instead of <code class="notranslate">131072.0</code> (stemming from the output values at indices <code class="notranslate">&gt;= 2**16</code> being <code class="notranslate">0</code>)</p>
<p dir="auto">Colab: <a href="https://colab.research.google.com/drive/1CZiKKWkI96SAVRJ5kMkT5kmMunipJhdx" rel="nofollow">https://colab.research.google.com/drive/1CZiKKWkI96SAVRJ5kMkT5kmMunipJhdx</a></p>
<p dir="auto">Some observations:</p>
<ul dir="auto">
<li>This occurs for batch dimensions greater than or equal to 2<sup>16</sup>.</li>
<li>This only occurs on the GPU. The CPU version seems fine.</li>
<li>This only occurs for FP32. FP64 seems fine.</li>
<li>In TensorFlow 2.0, the bug only manifests on the first run (as shown in the colab example). Subsequent runs produce the correct result.</li>
</ul>