<p dir="auto"><strong>System information</strong></p>
<ul dir="auto">
<li>Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes</li>
<li>OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 18.04</li>
<li>Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:</li>
<li>TensorFlow installed from (source or binary): binary</li>
<li>TensorFlow version (use command below): v1.14.0-rc1-22-gaf24dc91b5 1.14.0</li>
<li>Python version: 3.6.7</li>
<li>GCC/Compiler version (if compiling from source): 8.2.0</li>
<li>CUDA/cuDNN version: cuda-10.0</li>
<li>GPU model and memory:</li>
</ul>
<p dir="auto"><strong>Describe the current behavior</strong><br>
tf.matmul on tensorflow-gpu gave wrong results. Here is a simplified version of the code.</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt

r = [[ 1.0, 0.0],    [0.0, 1.0]]

x, y = np.meshgrid(list(range(400)), list(range(400)))
coords = np.stack([x,y],-1).reshape((400,400,2,1))
coords = tf.convert_to_tensor(coords,dtype=tf.float32)

r1 = tf.constant(r)

newCoords = tf.matmul(r1, coords)

sess = tf.Session()
ret = sess.run(newCoords,feed_dict={r1:r})

plt.matshow(ret[:,:,0,0])
plt.show()"><pre class="notranslate"><code class="notranslate">import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt

r = [[ 1.0, 0.0],    [0.0, 1.0]]

x, y = np.meshgrid(list(range(400)), list(range(400)))
coords = np.stack([x,y],-1).reshape((400,400,2,1))
coords = tf.convert_to_tensor(coords,dtype=tf.float32)

r1 = tf.constant(r)

newCoords = tf.matmul(r1, coords)

sess = tf.Session()
ret = sess.run(newCoords,feed_dict={r1:r})

plt.matshow(ret[:,:,0,0])
plt.show()
</code></pre></div>
<p dir="auto">When I ran it on my tensorflow-gpu, here is the result:<br>
<a target="_blank" rel="noopener noreferrer nofollow" href="https://gist.githubusercontent.com/sWizad/3a25d6559ea3308e5cc2731519635c32/raw/2479341ed43fb8e69c94eda3a1362d5cbde7d2d7/Figure_1.png"><img src="https://gist.githubusercontent.com/sWizad/3a25d6559ea3308e5cc2731519635c32/raw/2479341ed43fb8e69c94eda3a1362d5cbde7d2d7/Figure_1.png" alt="bug" style="max-width: 100%;"></a><br>
Looks like it stops computing halfway through and gave the rest 0 as a result.</p>
<p dir="auto"><strong>Describe the expected behavior</strong><br>
Here is the result with CPU:<br>
<a target="_blank" rel="noopener noreferrer nofollow" href="https://user-images.githubusercontent.com/45821224/62137176-08631480-b2d5-11e9-8b8e-6348e8b206e7.png"><img src="https://user-images.githubusercontent.com/45821224/62137176-08631480-b2d5-11e9-8b8e-6348e8b206e7.png" alt="CPU" style="max-width: 100%;"></a></p>
<hr>
<p dir="auto">Below is my old post. Initially, I thought the problem was related to TFRecord, but seems like this problem occurs without even using tfrecord too. For completeness, I keep the old example code with tfrecord.</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt

def parser(serialized_example):
      fs = tf.io.parse_single_example(
          serialized_example,
          features={ &quot;r&quot;: tf.FixedLenFeature([4], tf.float32) })
      fs[&quot;r&quot;] = tf.reshape(fs[&quot;r&quot;], [2, 2])
      return fs

r = [[ 1.0, 0.0],[0.0, 1.0]]

with tf.io.TFRecordWriter(&quot;cc.test&quot;) as tfrecord_writer:
    feature = {&quot;r&quot;: tf.train.Feature(float_list=tf.train.FloatList(value=np.array(r).flatten() ))}
    example = tf.train.Example(features=tf.train.Features(feature=feature))
    tfrecord_writer.write(example.SerializeToString())
dataset = tf.data.TFRecordDataset([&quot;cc.test&quot;])
dataset = dataset.map(parser).repeat().make_one_shot_iterator()
features = dataset.get_next()

x, y = tf.meshgrid(list(range(400)), list(range(400)))
coords = tf.stack([x, y], -1)     #(h,w,2)
coords = tf.expand_dims(tf.cast(coords,tf.float32),-1) #(h,w,2,1)

r1 = features[&quot;r&quot;]
r2 = tf.constant(r)

newCoords = tf.matmul(r1, coords)

sess = tf.Session()
ret = sess.run(newCoords[:,:,0,0])
plt.matshow(ret)
plt.show()"><pre class="notranslate"><code class="notranslate">import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt

def parser(serialized_example):
      fs = tf.io.parse_single_example(
          serialized_example,
          features={ "r": tf.FixedLenFeature([4], tf.float32) })
      fs["r"] = tf.reshape(fs["r"], [2, 2])
      return fs

r = [[ 1.0, 0.0],[0.0, 1.0]]

with tf.io.TFRecordWriter("cc.test") as tfrecord_writer:
    feature = {"r": tf.train.Feature(float_list=tf.train.FloatList(value=np.array(r).flatten() ))}
    example = tf.train.Example(features=tf.train.Features(feature=feature))
    tfrecord_writer.write(example.SerializeToString())
dataset = tf.data.TFRecordDataset(["cc.test"])
dataset = dataset.map(parser).repeat().make_one_shot_iterator()
features = dataset.get_next()

x, y = tf.meshgrid(list(range(400)), list(range(400)))
coords = tf.stack([x, y], -1)     #(h,w,2)
coords = tf.expand_dims(tf.cast(coords,tf.float32),-1) #(h,w,2,1)

r1 = features["r"]
r2 = tf.constant(r)

newCoords = tf.matmul(r1, coords)

sess = tf.Session()
ret = sess.run(newCoords[:,:,0,0])
plt.matshow(ret)
plt.show()
</code></pre></div>
<p dir="auto">The code will create "cc.test" file to save the variable <code class="notranslate">r</code> and load it as <code class="notranslate">r1</code>. Then matmul <code class="notranslate">r1</code> with some big varibles.</p>
<p dir="auto">Can't compile tensorflow on Ubuntu with Skylake CPU (i7-7820X). I get the following error:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="ERROR: /home/Dmitry/Private/Kaggle/tensorflow/tensorflow/core/kernels/BUILD:1369:1: C++ compilation of rule '//tensorflow/core/kernels:range_sampler' failed (Exit 1): crosstool_wrapper_driver_is_not_gcc failed: error executing command 
  (cd /home/Dmitry/.cache/bazel/_bazel_Dmitry/a3a733ca304216612b7ed32f720401f1/execroot/org_tensorflow &amp;&amp; \
  exec env - \
    CUDA_TOOLKIT_PATH=/usr/local/cuda \
    CUDNN_INSTALL_PATH=/usr/local/cuda-8.0 \
    GCC_HOST_COMPILER_PATH=/usr/bin/gcc-4.9 \
    PWD=/proc/self/cwd \
    PYTHON_BIN_PATH=/usr/bin/python \
    PYTHON_LIB_PATH=/usr/local/lib/python2.7/dist-packages \
    TF_CUDA_CLANG=0 \
    TF_CUDA_COMPUTE_CAPABILITIES=6.1 \
    TF_CUDA_VERSION=8.0 \
    TF_CUDNN_VERSION=6 \
    TF_NEED_CUDA=1 \
    TF_NEED_OPENCL=0 \
  external/local_config_cuda/crosstool/clang/bin/crosstool_wrapper_driver_is_not_gcc -U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -fPIE -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 -DNDEBUG -ffunction-sections -fdata-sections '-march=native' '-std=c++11' '-march=native' '-fabi-version=0' '-D_GLIBCXX_USE_CXX11_ABI=0' -MD -MF bazel-out/local_linux-opt/bin/tensorflow/core/kernels/_objs/range_sampler/tensorflow/core/kernels/range_sampler.pic.d '-frandom-seed=bazel-out/local_linux-opt/bin/tensorflow/core/kernels/_objs/range_sampler/tensorflow/core/kernels/range_sampler.pic.o' -fPIC -DEIGEN_MPL2_ONLY -DTENSORFLOW_USE_JEMALLOC -DSNAPPY -iquote . -iquote bazel-out/local_linux-opt/genfiles -iquote external/nsync -iquote bazel-out/local_linux-opt/genfiles/external/nsync -iquote external/bazel_tools -iquote bazel-out/local_linux-opt/genfiles/external/bazel_tools -iquote external/jemalloc -iquote bazel-out/local_linux-opt/genfiles/external/jemalloc -iquote external/protobuf_archive -iquote bazel-out/local_linux-opt/genfiles/external/protobuf_archive -iquote external/eigen_archive -iquote bazel-out/local_linux-opt/genfiles/external/eigen_archive -iquote external/local_config_sycl -iquote bazel-out/local_linux-opt/genfiles/external/local_config_sycl -iquote external/gif_archive -iquote bazel-out/local_linux-opt/genfiles/external/gif_archive -iquote external/jpeg -iquote bazel-out/local_linux-opt/genfiles/external/jpeg -iquote external/com_googlesource_code_re2 -iquote bazel-out/local_linux-opt/genfiles/external/com_googlesource_code_re2 -iquote external/farmhash_archive -iquote bazel-out/local_linux-opt/genfiles/external/farmhash_archive -iquote external/fft2d -iquote bazel-out/local_linux-opt/genfiles/external/fft2d -iquote external/highwayhash -iquote bazel-out/local_linux-opt/genfiles/external/highwayhash -iquote external/png_archive -iquote bazel-out/local_linux-opt/genfiles/external/png_archive -iquote external/zlib_archive -iquote bazel-out/local_linux-opt/genfiles/external/zlib_archive -iquote external/snappy -iquote bazel-out/local_linux-opt/genfiles/external/snappy -isystem external/nsync/public -isystem bazel-out/local_linux-opt/genfiles/external/nsync/public -isystem external/bazel_tools/tools/cpp/gcc3 -isystem external/jemalloc/include -isystem bazel-out/local_linux-opt/genfiles/external/jemalloc/include -isystem external/protobuf_archive/src -isystem bazel-out/local_linux-opt/genfiles/external/protobuf_archive/src -isystem external/eigen_archive -isystem bazel-out/local_linux-opt/genfiles/external/eigen_archive -isystem external/gif_archive/lib -isystem bazel-out/local_linux-opt/genfiles/external/gif_archive/lib -isystem external/farmhash_archive/src -isystem bazel-out/local_linux-opt/genfiles/external/farmhash_archive/src -isystem external/png_archive -isystem bazel-out/local_linux-opt/genfiles/external/png_archive -isystem external/zlib_archive -isystem bazel-out/local_linux-opt/genfiles/external/zlib_archive -no-canonical-prefixes -Wno-builtin-macro-redefined '-D__DATE__=&quot;redacted&quot;' '-D__TIMESTAMP__=&quot;redacted&quot;' '-D__TIME__=&quot;redacted&quot;' -fno-canonical-system-headers -c tensorflow/core/kernels/range_sampler.cc -o bazel-out/local_linux-opt/bin/tensorflow/core/kernels/_objs/range_sampler/tensorflow/core/kernels/range_sampler.pic.o)
In file included from external/eigen_archive/unsupported/Eigen/CXX11/../../../Eigen/Core:378:0,
                 from external/eigen_archive/unsupported/Eigen/CXX11/Tensor:14,
                 from ./third_party/eigen3/unsupported/Eigen/CXX11/Tensor:1,
                 from ./tensorflow/core/lib/random/random_distributions.h:27,
                 from ./tensorflow/core/lib/random/simple_philox.h:24,
                 from ./tensorflow/core/lib/random/distribution_sampler.h:38,
                 from ./tensorflow/core/kernels/range_sampler.h:23,
                 from tensorflow/core/kernels/range_sampler.cc:16:
external/eigen_archive/unsupported/Eigen/CXX11/../../../Eigen/src/Core/arch/AVX512/PacketMath.h: In function 'Packet Eigen::internal::ploaddup(const typename Eigen::internal::unpacket_traits&lt;Packet&gt;::type*) [with Packet = __vector(16) float; typename Eigen::internal::unpacket_traits&lt;Packet&gt;::type = float]':
external/eigen_archive/unsupported/Eigen/CXX11/../../../Eigen/src/Core/arch/AVX512/PacketMath.h:469:77: error: '_mm512_castsi512_ps' was not declared in this scope
   __m512 even_elements = _mm512_castsi512_ps(_mm512_cvtepu32_epi64(low_half));
                                                                             ^
external/eigen_archive/unsupported/Eigen/CXX11/../../../Eigen/src/Core/arch/AVX512/PacketMath.h: In function 'Packet Eigen::internal::ploaddup(const typename Eigen::internal::unpacket_traits&lt;Packet&gt;::type*) [with Packet = __vector(8) double; typename Eigen::internal::unpacket_traits&lt;Packet&gt;::type = double]':
external/eigen_archive/unsupported/Eigen/CXX11/../../../Eigen/src/Core/arch/AVX512/PacketMath.h:478:56: error: '_mm512_insertf64x2' was not declared in this scope
   x = _mm512_insertf64x2(x, _mm_loaddup_pd(&amp;from[0]), 0);
                                                        ^
In file included from ./tensorflow/core/platform/default/logging.h:24:0,
                 from ./tensorflow/core/platform/logging.h:25,
                 from ./tensorflow/core/lib/core/status.h:25,
                 from ./tensorflow/core/kernels/range_sampler.h:21,
                 from tensorflow/core/kernels/range_sampler.cc:16:
./tensorflow/core/platform/default/logging.h: In instantiation of 'std::string* tensorflow::internal::Check_LEImpl(const T1&amp;, const T2&amp;, const char*) [with T1 = long unsigned int; T2 = long long int; std::string = std::basic_string&lt;char&gt;]':
tensorflow/core/kernels/range_sampler.cc:86:5:   required from here
./tensorflow/core/platform/default/logging.h:230:35: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
 TF_DEFINE_CHECK_OP_IMPL(Check_LE, &lt;= )
                                   ^
./tensorflow/core/platform/macros.h:79:29: note: in definition of macro 'TF_PREDICT_TRUE'
 #define TF_PREDICT_TRUE(x) (x)
                             ^
./tensorflow/core/platform/default/logging.h:230:1: note: in expansion of macro 'TF_DEFINE_CHECK_OP_IMPL'
 TF_DEFINE_CHECK_OP_IMPL(Check_LE, &lt;= )
 ^
./tensorflow/core/platform/default/logging.h: In instantiation of 'std::string* tensorflow::internal::Check_EQImpl(const T1&amp;, const T2&amp;, const char*) [with T1 = long long int; T2 = long unsigned int; std::string = std::basic_string&lt;char&gt;]':
tensorflow/core/kernels/range_sampler.cc:244:3:   required from here
./tensorflow/core/platform/default/logging.h:228:25: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
                         == )  // Compilation error with CHECK_EQ(NULL, x)?
                         ^
./tensorflow/core/platform/macros.h:79:29: note: in definition of macro 'TF_PREDICT_TRUE'
 #define TF_PREDICT_TRUE(x) (x)
                             ^
./tensorflow/core/platform/default/logging.h:227:1: note: in expansion of macro 'TF_DEFINE_CHECK_OP_IMPL'
 TF_DEFINE_CHECK_OP_IMPL(Check_EQ,
 ^
Target //tensorflow/tools/pip_package:build_pip_package failed to build
INFO: Elapsed time: 1.894s, Critical Path: 1.15s
FAILED: Build did NOT complete successfully"><pre class="notranslate"><code class="notranslate">ERROR: /home/Dmitry/Private/Kaggle/tensorflow/tensorflow/core/kernels/BUILD:1369:1: C++ compilation of rule '//tensorflow/core/kernels:range_sampler' failed (Exit 1): crosstool_wrapper_driver_is_not_gcc failed: error executing command 
  (cd /home/Dmitry/.cache/bazel/_bazel_Dmitry/a3a733ca304216612b7ed32f720401f1/execroot/org_tensorflow &amp;&amp; \
  exec env - \
    CUDA_TOOLKIT_PATH=/usr/local/cuda \
    CUDNN_INSTALL_PATH=/usr/local/cuda-8.0 \
    GCC_HOST_COMPILER_PATH=/usr/bin/gcc-4.9 \
    PWD=/proc/self/cwd \
    PYTHON_BIN_PATH=/usr/bin/python \
    PYTHON_LIB_PATH=/usr/local/lib/python2.7/dist-packages \
    TF_CUDA_CLANG=0 \
    TF_CUDA_COMPUTE_CAPABILITIES=6.1 \
    TF_CUDA_VERSION=8.0 \
    TF_CUDNN_VERSION=6 \
    TF_NEED_CUDA=1 \
    TF_NEED_OPENCL=0 \
  external/local_config_cuda/crosstool/clang/bin/crosstool_wrapper_driver_is_not_gcc -U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -fPIE -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 -DNDEBUG -ffunction-sections -fdata-sections '-march=native' '-std=c++11' '-march=native' '-fabi-version=0' '-D_GLIBCXX_USE_CXX11_ABI=0' -MD -MF bazel-out/local_linux-opt/bin/tensorflow/core/kernels/_objs/range_sampler/tensorflow/core/kernels/range_sampler.pic.d '-frandom-seed=bazel-out/local_linux-opt/bin/tensorflow/core/kernels/_objs/range_sampler/tensorflow/core/kernels/range_sampler.pic.o' -fPIC -DEIGEN_MPL2_ONLY -DTENSORFLOW_USE_JEMALLOC -DSNAPPY -iquote . -iquote bazel-out/local_linux-opt/genfiles -iquote external/nsync -iquote bazel-out/local_linux-opt/genfiles/external/nsync -iquote external/bazel_tools -iquote bazel-out/local_linux-opt/genfiles/external/bazel_tools -iquote external/jemalloc -iquote bazel-out/local_linux-opt/genfiles/external/jemalloc -iquote external/protobuf_archive -iquote bazel-out/local_linux-opt/genfiles/external/protobuf_archive -iquote external/eigen_archive -iquote bazel-out/local_linux-opt/genfiles/external/eigen_archive -iquote external/local_config_sycl -iquote bazel-out/local_linux-opt/genfiles/external/local_config_sycl -iquote external/gif_archive -iquote bazel-out/local_linux-opt/genfiles/external/gif_archive -iquote external/jpeg -iquote bazel-out/local_linux-opt/genfiles/external/jpeg -iquote external/com_googlesource_code_re2 -iquote bazel-out/local_linux-opt/genfiles/external/com_googlesource_code_re2 -iquote external/farmhash_archive -iquote bazel-out/local_linux-opt/genfiles/external/farmhash_archive -iquote external/fft2d -iquote bazel-out/local_linux-opt/genfiles/external/fft2d -iquote external/highwayhash -iquote bazel-out/local_linux-opt/genfiles/external/highwayhash -iquote external/png_archive -iquote bazel-out/local_linux-opt/genfiles/external/png_archive -iquote external/zlib_archive -iquote bazel-out/local_linux-opt/genfiles/external/zlib_archive -iquote external/snappy -iquote bazel-out/local_linux-opt/genfiles/external/snappy -isystem external/nsync/public -isystem bazel-out/local_linux-opt/genfiles/external/nsync/public -isystem external/bazel_tools/tools/cpp/gcc3 -isystem external/jemalloc/include -isystem bazel-out/local_linux-opt/genfiles/external/jemalloc/include -isystem external/protobuf_archive/src -isystem bazel-out/local_linux-opt/genfiles/external/protobuf_archive/src -isystem external/eigen_archive -isystem bazel-out/local_linux-opt/genfiles/external/eigen_archive -isystem external/gif_archive/lib -isystem bazel-out/local_linux-opt/genfiles/external/gif_archive/lib -isystem external/farmhash_archive/src -isystem bazel-out/local_linux-opt/genfiles/external/farmhash_archive/src -isystem external/png_archive -isystem bazel-out/local_linux-opt/genfiles/external/png_archive -isystem external/zlib_archive -isystem bazel-out/local_linux-opt/genfiles/external/zlib_archive -no-canonical-prefixes -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -fno-canonical-system-headers -c tensorflow/core/kernels/range_sampler.cc -o bazel-out/local_linux-opt/bin/tensorflow/core/kernels/_objs/range_sampler/tensorflow/core/kernels/range_sampler.pic.o)
In file included from external/eigen_archive/unsupported/Eigen/CXX11/../../../Eigen/Core:378:0,
                 from external/eigen_archive/unsupported/Eigen/CXX11/Tensor:14,
                 from ./third_party/eigen3/unsupported/Eigen/CXX11/Tensor:1,
                 from ./tensorflow/core/lib/random/random_distributions.h:27,
                 from ./tensorflow/core/lib/random/simple_philox.h:24,
                 from ./tensorflow/core/lib/random/distribution_sampler.h:38,
                 from ./tensorflow/core/kernels/range_sampler.h:23,
                 from tensorflow/core/kernels/range_sampler.cc:16:
external/eigen_archive/unsupported/Eigen/CXX11/../../../Eigen/src/Core/arch/AVX512/PacketMath.h: In function 'Packet Eigen::internal::ploaddup(const typename Eigen::internal::unpacket_traits&lt;Packet&gt;::type*) [with Packet = __vector(16) float; typename Eigen::internal::unpacket_traits&lt;Packet&gt;::type = float]':
external/eigen_archive/unsupported/Eigen/CXX11/../../../Eigen/src/Core/arch/AVX512/PacketMath.h:469:77: error: '_mm512_castsi512_ps' was not declared in this scope
   __m512 even_elements = _mm512_castsi512_ps(_mm512_cvtepu32_epi64(low_half));
                                                                             ^
external/eigen_archive/unsupported/Eigen/CXX11/../../../Eigen/src/Core/arch/AVX512/PacketMath.h: In function 'Packet Eigen::internal::ploaddup(const typename Eigen::internal::unpacket_traits&lt;Packet&gt;::type*) [with Packet = __vector(8) double; typename Eigen::internal::unpacket_traits&lt;Packet&gt;::type = double]':
external/eigen_archive/unsupported/Eigen/CXX11/../../../Eigen/src/Core/arch/AVX512/PacketMath.h:478:56: error: '_mm512_insertf64x2' was not declared in this scope
   x = _mm512_insertf64x2(x, _mm_loaddup_pd(&amp;from[0]), 0);
                                                        ^
In file included from ./tensorflow/core/platform/default/logging.h:24:0,
                 from ./tensorflow/core/platform/logging.h:25,
                 from ./tensorflow/core/lib/core/status.h:25,
                 from ./tensorflow/core/kernels/range_sampler.h:21,
                 from tensorflow/core/kernels/range_sampler.cc:16:
./tensorflow/core/platform/default/logging.h: In instantiation of 'std::string* tensorflow::internal::Check_LEImpl(const T1&amp;, const T2&amp;, const char*) [with T1 = long unsigned int; T2 = long long int; std::string = std::basic_string&lt;char&gt;]':
tensorflow/core/kernels/range_sampler.cc:86:5:   required from here
./tensorflow/core/platform/default/logging.h:230:35: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
 TF_DEFINE_CHECK_OP_IMPL(Check_LE, &lt;= )
                                   ^
./tensorflow/core/platform/macros.h:79:29: note: in definition of macro 'TF_PREDICT_TRUE'
 #define TF_PREDICT_TRUE(x) (x)
                             ^
./tensorflow/core/platform/default/logging.h:230:1: note: in expansion of macro 'TF_DEFINE_CHECK_OP_IMPL'
 TF_DEFINE_CHECK_OP_IMPL(Check_LE, &lt;= )
 ^
./tensorflow/core/platform/default/logging.h: In instantiation of 'std::string* tensorflow::internal::Check_EQImpl(const T1&amp;, const T2&amp;, const char*) [with T1 = long long int; T2 = long unsigned int; std::string = std::basic_string&lt;char&gt;]':
tensorflow/core/kernels/range_sampler.cc:244:3:   required from here
./tensorflow/core/platform/default/logging.h:228:25: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
                         == )  // Compilation error with CHECK_EQ(NULL, x)?
                         ^
./tensorflow/core/platform/macros.h:79:29: note: in definition of macro 'TF_PREDICT_TRUE'
 #define TF_PREDICT_TRUE(x) (x)
                             ^
./tensorflow/core/platform/default/logging.h:227:1: note: in expansion of macro 'TF_DEFINE_CHECK_OP_IMPL'
 TF_DEFINE_CHECK_OP_IMPL(Check_EQ,
 ^
Target //tensorflow/tools/pip_package:build_pip_package failed to build
INFO: Elapsed time: 1.894s, Critical Path: 1.15s
FAILED: Build did NOT complete successfully
</code></pre></div>
<p dir="auto">OR the following error:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="ERROR: /home/Dmitry/Private/Kaggle/tensorflow/tensorflow/core/BUILD:1299:1: C++ compilation of rule '//tensorflow/core:lib_internal' failed (Exit 1): crosstool_wrapper_driver_is_not_gcc failed: error executing command 
  (cd /home/Dmitry/.cache/bazel/_bazel_Dmitry/a3a733ca304216612b7ed32f720401f1/execroot/org_tensorflow &amp;&amp; \
  exec env - \
    CUDA_TOOLKIT_PATH=/usr/local/cuda \
    CUDNN_INSTALL_PATH=/usr/local/cuda-8.0 \
    GCC_HOST_COMPILER_PATH=/usr/bin/gcc-4.9 \
    PWD=/proc/self/cwd \
    PYTHON_BIN_PATH=/usr/bin/python3 \
    PYTHON_LIB_PATH=/usr/local/lib/python3.5/dist-packages \
    TF_CUDA_CLANG=0 \
    TF_CUDA_COMPUTE_CAPABILITIES=6.1 \
    TF_CUDA_VERSION=8.0 \
    TF_CUDNN_VERSION=6 \
    TF_NEED_CUDA=1 \
    TF_NEED_OPENCL=0 \
  external/local_config_cuda/crosstool/clang/bin/crosstool_wrapper_driver_is_not_gcc -U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -fPIE -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 -DNDEBUG -ffunction-sections -fdata-sections '-march=native' '-std=c++11' '-march=native' '-fabi-version=0' '-D_GLIBCXX_USE_CXX11_ABI=0' -MD -MF bazel-out/local_linux-py3-opt/bin/tensorflow/core/_objs/lib_internal/tensorflow/core/lib/random/distribution_sampler.pic.d '-frandom-seed=bazel-out/local_linux-py3-opt/bin/tensorflow/core/_objs/lib_internal/tensorflow/core/lib/random/distribution_sampler.pic.o' -fPIC -DEIGEN_MPL2_ONLY -DTENSORFLOW_USE_JEMALLOC -DSNAPPY -iquote . -iquote bazel-out/local_linux-py3-opt/genfiles -iquote external/nsync -iquote bazel-out/local_linux-py3-opt/genfiles/external/nsync -iquote external/bazel_tools -iquote bazel-out/local_linux-py3-opt/genfiles/external/bazel_tools -iquote external/jemalloc -iquote bazel-out/local_linux-py3-opt/genfiles/external/jemalloc -iquote external/protobuf_archive -iquote bazel-out/local_linux-py3-opt/genfiles/external/protobuf_archive -iquote external/eigen_archive -iquote bazel-out/local_linux-py3-opt/genfiles/external/eigen_archive -iquote external/local_config_sycl -iquote bazel-out/local_linux-py3-opt/genfiles/external/local_config_sycl -iquote external/gif_archive -iquote bazel-out/local_linux-py3-opt/genfiles/external/gif_archive -iquote external/jpeg -iquote bazel-out/local_linux-py3-opt/genfiles/external/jpeg -iquote external/com_googlesource_code_re2 -iquote bazel-out/local_linux-py3-opt/genfiles/external/com_googlesource_code_re2 -iquote external/farmhash_archive -iquote bazel-out/local_linux-py3-opt/genfiles/external/farmhash_archive -iquote external/fft2d -iquote bazel-out/local_linux-py3-opt/genfiles/external/fft2d -iquote external/highwayhash -iquote bazel-out/local_linux-py3-opt/genfiles/external/highwayhash -iquote external/png_archive -iquote bazel-out/local_linux-py3-opt/genfiles/external/png_archive -iquote external/zlib_archive -iquote bazel-out/local_linux-py3-opt/genfiles/external/zlib_archive -iquote external/snappy -iquote bazel-out/local_linux-py3-opt/genfiles/external/snappy -isystem external/nsync/public -isystem bazel-out/local_linux-py3-opt/genfiles/external/nsync/public -isystem external/bazel_tools/tools/cpp/gcc3 -isystem external/jemalloc/include -isystem bazel-out/local_linux-py3-opt/genfiles/external/jemalloc/include -isystem external/protobuf_archive/src -isystem bazel-out/local_linux-py3-opt/genfiles/external/protobuf_archive/src -isystem external/eigen_archive -isystem bazel-out/local_linux-py3-opt/genfiles/external/eigen_archive -isystem external/gif_archive/lib -isystem bazel-out/local_linux-py3-opt/genfiles/external/gif_archive/lib -isystem external/farmhash_archive/src -isystem bazel-out/local_linux-py3-opt/genfiles/external/farmhash_archive/src -isystem external/png_archive -isystem bazel-out/local_linux-py3-opt/genfiles/external/png_archive -isystem external/zlib_archive -isystem bazel-out/local_linux-py3-opt/genfiles/external/zlib_archive -DEIGEN_AVOID_STL_ARRAY -Iexternal/gemmlowp -Wno-sign-compare -fno-exceptions '-ftemplate-depth=900' '-DGOOGLE_CUDA=1' -msse3 -pthread -no-canonical-prefixes -Wno-builtin-macro-redefined '-D__DATE__=&quot;redacted&quot;' '-D__TIMESTAMP__=&quot;redacted&quot;' '-D__TIME__=&quot;redacted&quot;' -fno-canonical-system-headers -c tensorflow/core/lib/random/distribution_sampler.cc -o bazel-out/local_linux-py3-opt/bin/tensorflow/core/_objs/lib_internal/tensorflow/core/lib/random/distribution_sampler.pic.o)
In file included from external/eigen_archive/unsupported/Eigen/CXX11/../../../Eigen/Core:378:0,
                 from external/eigen_archive/unsupported/Eigen/CXX11/Tensor:14,
                 from ./third_party/eigen3/unsupported/Eigen/CXX11/Tensor:1,
                 from ./tensorflow/core/lib/random/random_distributions.h:27,
                 from ./tensorflow/core/lib/random/simple_philox.h:24,
                 from ./tensorflow/core/lib/random/distribution_sampler.h:38,
                 from tensorflow/core/lib/random/distribution_sampler.cc:16:
external/eigen_archive/unsupported/Eigen/CXX11/../../../Eigen/src/Core/arch/AVX512/PacketMath.h: In function 'Packet Eigen::internal::ploaddup(const typename Eigen::internal::unpacket_traits&lt;Packet&gt;::type*) [with Packet = __vector(16) float; typename Eigen::internal::unpacket_traits&lt;Packet&gt;::type = float]':
external/eigen_archive/unsupported/Eigen/CXX11/../../../Eigen/src/Core/arch/AVX512/PacketMath.h:469:77: error: '_mm512_castsi512_ps' was not declared in this scope
   __m512 even_elements = _mm512_castsi512_ps(_mm512_cvtepu32_epi64(low_half));
                                                                             ^
external/eigen_archive/unsupported/Eigen/CXX11/../../../Eigen/src/Core/arch/AVX512/PacketMath.h: In function 'Packet Eigen::internal::ploaddup(const typename Eigen::internal::unpacket_traits&lt;Packet&gt;::type*) [with Packet = __vector(8) double; typename Eigen::internal::unpacket_traits&lt;Packet&gt;::type = double]':
external/eigen_archive/unsupported/Eigen/CXX11/../../../Eigen/src/Core/arch/AVX512/PacketMath.h:478:56: error: '_mm512_insertf64x2' was not declared in this scope
   x = _mm512_insertf64x2(x, _mm_loaddup_pd(&amp;from[0]), 0);
                                                        ^
Target //tensorflow/tools/pip_package:build_pip_package failed to build
INFO: Elapsed time: 183.759s, Critical Path: 16.18s
FAILED: Build did NOT complete successfully"><pre class="notranslate"><code class="notranslate">ERROR: /home/Dmitry/Private/Kaggle/tensorflow/tensorflow/core/BUILD:1299:1: C++ compilation of rule '//tensorflow/core:lib_internal' failed (Exit 1): crosstool_wrapper_driver_is_not_gcc failed: error executing command 
  (cd /home/Dmitry/.cache/bazel/_bazel_Dmitry/a3a733ca304216612b7ed32f720401f1/execroot/org_tensorflow &amp;&amp; \
  exec env - \
    CUDA_TOOLKIT_PATH=/usr/local/cuda \
    CUDNN_INSTALL_PATH=/usr/local/cuda-8.0 \
    GCC_HOST_COMPILER_PATH=/usr/bin/gcc-4.9 \
    PWD=/proc/self/cwd \
    PYTHON_BIN_PATH=/usr/bin/python3 \
    PYTHON_LIB_PATH=/usr/local/lib/python3.5/dist-packages \
    TF_CUDA_CLANG=0 \
    TF_CUDA_COMPUTE_CAPABILITIES=6.1 \
    TF_CUDA_VERSION=8.0 \
    TF_CUDNN_VERSION=6 \
    TF_NEED_CUDA=1 \
    TF_NEED_OPENCL=0 \
  external/local_config_cuda/crosstool/clang/bin/crosstool_wrapper_driver_is_not_gcc -U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -fPIE -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 -DNDEBUG -ffunction-sections -fdata-sections '-march=native' '-std=c++11' '-march=native' '-fabi-version=0' '-D_GLIBCXX_USE_CXX11_ABI=0' -MD -MF bazel-out/local_linux-py3-opt/bin/tensorflow/core/_objs/lib_internal/tensorflow/core/lib/random/distribution_sampler.pic.d '-frandom-seed=bazel-out/local_linux-py3-opt/bin/tensorflow/core/_objs/lib_internal/tensorflow/core/lib/random/distribution_sampler.pic.o' -fPIC -DEIGEN_MPL2_ONLY -DTENSORFLOW_USE_JEMALLOC -DSNAPPY -iquote . -iquote bazel-out/local_linux-py3-opt/genfiles -iquote external/nsync -iquote bazel-out/local_linux-py3-opt/genfiles/external/nsync -iquote external/bazel_tools -iquote bazel-out/local_linux-py3-opt/genfiles/external/bazel_tools -iquote external/jemalloc -iquote bazel-out/local_linux-py3-opt/genfiles/external/jemalloc -iquote external/protobuf_archive -iquote bazel-out/local_linux-py3-opt/genfiles/external/protobuf_archive -iquote external/eigen_archive -iquote bazel-out/local_linux-py3-opt/genfiles/external/eigen_archive -iquote external/local_config_sycl -iquote bazel-out/local_linux-py3-opt/genfiles/external/local_config_sycl -iquote external/gif_archive -iquote bazel-out/local_linux-py3-opt/genfiles/external/gif_archive -iquote external/jpeg -iquote bazel-out/local_linux-py3-opt/genfiles/external/jpeg -iquote external/com_googlesource_code_re2 -iquote bazel-out/local_linux-py3-opt/genfiles/external/com_googlesource_code_re2 -iquote external/farmhash_archive -iquote bazel-out/local_linux-py3-opt/genfiles/external/farmhash_archive -iquote external/fft2d -iquote bazel-out/local_linux-py3-opt/genfiles/external/fft2d -iquote external/highwayhash -iquote bazel-out/local_linux-py3-opt/genfiles/external/highwayhash -iquote external/png_archive -iquote bazel-out/local_linux-py3-opt/genfiles/external/png_archive -iquote external/zlib_archive -iquote bazel-out/local_linux-py3-opt/genfiles/external/zlib_archive -iquote external/snappy -iquote bazel-out/local_linux-py3-opt/genfiles/external/snappy -isystem external/nsync/public -isystem bazel-out/local_linux-py3-opt/genfiles/external/nsync/public -isystem external/bazel_tools/tools/cpp/gcc3 -isystem external/jemalloc/include -isystem bazel-out/local_linux-py3-opt/genfiles/external/jemalloc/include -isystem external/protobuf_archive/src -isystem bazel-out/local_linux-py3-opt/genfiles/external/protobuf_archive/src -isystem external/eigen_archive -isystem bazel-out/local_linux-py3-opt/genfiles/external/eigen_archive -isystem external/gif_archive/lib -isystem bazel-out/local_linux-py3-opt/genfiles/external/gif_archive/lib -isystem external/farmhash_archive/src -isystem bazel-out/local_linux-py3-opt/genfiles/external/farmhash_archive/src -isystem external/png_archive -isystem bazel-out/local_linux-py3-opt/genfiles/external/png_archive -isystem external/zlib_archive -isystem bazel-out/local_linux-py3-opt/genfiles/external/zlib_archive -DEIGEN_AVOID_STL_ARRAY -Iexternal/gemmlowp -Wno-sign-compare -fno-exceptions '-ftemplate-depth=900' '-DGOOGLE_CUDA=1' -msse3 -pthread -no-canonical-prefixes -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -fno-canonical-system-headers -c tensorflow/core/lib/random/distribution_sampler.cc -o bazel-out/local_linux-py3-opt/bin/tensorflow/core/_objs/lib_internal/tensorflow/core/lib/random/distribution_sampler.pic.o)
In file included from external/eigen_archive/unsupported/Eigen/CXX11/../../../Eigen/Core:378:0,
                 from external/eigen_archive/unsupported/Eigen/CXX11/Tensor:14,
                 from ./third_party/eigen3/unsupported/Eigen/CXX11/Tensor:1,
                 from ./tensorflow/core/lib/random/random_distributions.h:27,
                 from ./tensorflow/core/lib/random/simple_philox.h:24,
                 from ./tensorflow/core/lib/random/distribution_sampler.h:38,
                 from tensorflow/core/lib/random/distribution_sampler.cc:16:
external/eigen_archive/unsupported/Eigen/CXX11/../../../Eigen/src/Core/arch/AVX512/PacketMath.h: In function 'Packet Eigen::internal::ploaddup(const typename Eigen::internal::unpacket_traits&lt;Packet&gt;::type*) [with Packet = __vector(16) float; typename Eigen::internal::unpacket_traits&lt;Packet&gt;::type = float]':
external/eigen_archive/unsupported/Eigen/CXX11/../../../Eigen/src/Core/arch/AVX512/PacketMath.h:469:77: error: '_mm512_castsi512_ps' was not declared in this scope
   __m512 even_elements = _mm512_castsi512_ps(_mm512_cvtepu32_epi64(low_half));
                                                                             ^
external/eigen_archive/unsupported/Eigen/CXX11/../../../Eigen/src/Core/arch/AVX512/PacketMath.h: In function 'Packet Eigen::internal::ploaddup(const typename Eigen::internal::unpacket_traits&lt;Packet&gt;::type*) [with Packet = __vector(8) double; typename Eigen::internal::unpacket_traits&lt;Packet&gt;::type = double]':
external/eigen_archive/unsupported/Eigen/CXX11/../../../Eigen/src/Core/arch/AVX512/PacketMath.h:478:56: error: '_mm512_insertf64x2' was not declared in this scope
   x = _mm512_insertf64x2(x, _mm_loaddup_pd(&amp;from[0]), 0);
                                                        ^
Target //tensorflow/tools/pip_package:build_pip_package failed to build
INFO: Elapsed time: 183.759s, Critical Path: 16.18s
FAILED: Build did NOT complete successfully
</code></pre></div>
<p dir="auto">Compilation command:</p>
<p dir="auto"><code class="notranslate">bazel build --config=opt --config=cuda --cxxopt="-fabi-version=0" //tensorflow/tools/pip_package:build_pip_package --verbose_failures</code></p>
<p dir="auto">Is it possible to disable AVX512 but leave all other native optimization parameters?</p>