<ol dir="auto">
<li>Loads of operations in the embedding example are not supported on GPU. Some documentation   detailing this would be good.</li>
<li>I tried to get around the problem by using 'tf.ConfigProto(allow_soft_placement=True)'. However now I get 'Executor failed to create kernel. Invalid argument: AttrValue must not have reference type value of float_ref for attr 'tensor_type'</li>
</ol>
<p dir="auto">Thanks<br>
Abhishek</p>
<p dir="auto">Please go to Stack Overflow for help and support:</p>
<p dir="auto"><a href="https://stackoverflow.com/questions/tagged/tensorflow" rel="nofollow">https://stackoverflow.com/questions/tagged/tensorflow</a></p>
<p dir="auto">If you open a GitHub issue, here is our policy:</p>
<ol dir="auto">
<li>It must be a bug or a feature request.</li>
<li>The form below must be filled out.</li>
<li>It shouldn't be a TensorBoard issue. Those go <a href="https://github.com/tensorflow/tensorboard/issues">here</a>.</li>
</ol>
<p dir="auto"><strong>Here's why we have that policy</strong>: TensorFlow developers respond to issues. We want to focus on work that benefits the whole community, e.g., fixing bugs and adding features. Support only helps individuals. GitHub also notifies thousands of people when issues are filed. We want them to see you communicating an interesting problem, rather than being redirected to Stack Overflow.</p>
<hr>
<h3 dir="auto">System information</h3>
<ul dir="auto">
<li><strong>Have I written custom code (as opposed to using a stock example script provided in TensorFlow)</strong>: Custom code! You can find it here: <a href="https://github.com/NERSC/CosmoFlow/tree/master/SegFault">https://github.com/NERSC/CosmoFlow/tree/master/SegFault</a></li>
<li><strong>OS Platform and Distribution (e.g., Linux Ubuntu 16.04)</strong>: SUSE Linux 12.2</li>
<li><strong>TensorFlow installed from (source or binary)</strong>: Source</li>
<li><strong>TensorFlow version (use command below)</strong>: ('v1.3.0-rc1-3112-g65b6a75', '1.4.0-rc0') Note this is NOT compiled with the Intel MKL options.</li>
<li><strong>Python version</strong>: 2.7.13</li>
<li><strong>Bazel version (if compiling from source)</strong>: 0.6.0</li>
<li><strong>GCC/Compiler version (if compiling from source)</strong>: 4.8.5</li>
<li><strong>CUDA/cuDNN version</strong>: N/A</li>
<li><strong>GPU model and memory</strong>: N/A, Running on x86_64 Intel Haswell node</li>
<li><strong>Exact command to reproduce</strong>: See README in <a href="https://github.com/NERSC/CosmoFlow/tree/master/SegFault">https://github.com/NERSC/CosmoFlow/tree/master/SegFault</a></li>
</ul>
<h3 dir="auto">Describe the problem</h3>
<p dir="auto">A seg fault when training a tf.nn.conv3d with minibatch size more than 2 on a single Intel Haswell. The seg fault occurs at <a href="https://github.com/NERSC/CosmoFlow/blob/master/SegFault/CosmoNet.py#L187">line 187</a>.</p>
<h3 dir="auto">Source code / logs</h3>
<p dir="auto">GDB log: <a href="https://github.com/NERSC/CosmoFlow/blob/master/SegFault/gdbTrace.log">https://github.com/NERSC/CosmoFlow/blob/master/SegFault/gdbTrace.log</a><br>
It looks like some kind of cyclic dependency in Eigen::TensorEvaluator.</p>