<p dir="auto">Hello,</p>
<p dir="auto">I get the following errors when compiling:</p>
<p dir="auto">____[110 / 2,641] Compiling external/png_archive/libpng-1.2.53/pngerror.c<br>
external/protobuf/src/google/protobuf/util/internal/field_mask_utility.cc:47:14: warning: 'google::protobuf::util::Status google::protobuf::util::converter::{anonymous}::CreatePublicError(google::protobuf::util::error::Code, const string&amp;)' defined but not used [-Wunused-function]<br>
util::Status CreatePublicError(util::error::Code code,<br>
external/protobuf/src/google/protobuf/util/internal/field_mask_utility.cc:47:14: warning: 'google::protobuf::util::Status google::protobuf::util::converter::{anonymous}::CreatePublicError(google::protobuf::util::error::Code, const string&amp;)' defined but not used [-Wunused-function]<br>
util::Status CreatePublicError(util::error::Code code,<br>
external/protobuf/src/google/protobuf/util/internal/field_mask_utility.cc:47:14: warning: 'google::protobuf::util::Status google::protobuf::util::converter::{anonymous}::CreatePublicError(google::protobuf::util::error::Code, const string&amp;)' defined but not used [-Wunused-function]<br>
util::Status CreatePublicError(util::error::Code code,<br>
== )  // Compilation error with CHECK_EQ(NULL, x)?<br>
== )  // Compilation error with CHECK_EQ(NULL, x)?<br>
ERROR: /home/user/Downloads/tensorflow/tensorflow/stream_executor/BUILD:5:1: C++ compilation of rule '//tensorflow/stream_executor:stream_executor' failed: crosstool_wrapper_driver_is_not_gcc failed: error executing command<br>
tensorflow/stream_executor/cuda/cuda_dnn.cc:266:10: error: 'CUDNN_CONVOLUTION_FWD_ALGO_FFT_TILING' was not declared in this scope<br>
tensorflow/stream_executor/cuda/cuda_dnn.cc:284:10: error: 'CUDNN_CONVOLUTION_BWD_DATA_ALGO_FFT_TILING' was not declared in this scope<br>
tensorflow/stream_executor/cuda/cuda_dnn.cc:942:7: error: 'CUDNN_CONVOLUTION_FWD_ALGO_FFT_TILING' was not declared in this scope<br>
tensorflow/stream_executor/cuda/cuda_dnn.cc:947:4: error: no matching function for call to 'std::vector::assign()'<br>
tensorflow/stream_executor/cuda/cuda_dnn.cc:958:7: error: 'CUDNN_CONVOLUTION_BWD_DATA_ALGO_FFT_TILING' was not declared in this scope<br>
tensorflow/stream_executor/cuda/cuda_dnn.cc:963:4: error: no matching function for call to 'std::vector::assign()'<br>
tensorflow/stream_executor/cuda/cuda_dnn.cc:166:39: error: too many arguments to function</p>
<p dir="auto">Complete log of build: <a href="http://pastebin.com/NzpiwmGm" rel="nofollow">http://pastebin.com/NzpiwmGm</a></p>
<p dir="auto">The compilation command that I am using is:<br>
bazel build --verbose_failures --genrule_strategy=standalone --spawn_strategy=standalone -c opt --config=cuda //tensorflow/cc:tutorials_example_trainer &gt;&amp; build.log &amp;</p>
<p dir="auto">My systems information is as follows</p>
<h3 dir="auto">Environment info</h3>
<p dir="auto">Operating System:<br>
Distributor ID: Ubuntu<br>
Description:    Ubuntu 16.04.1 LTS<br>
Release:    16.04<br>
Codename:   xenial</p>
<p dir="auto">Installed version of CUDA and cuDNN:<br>
/usr/lib/x86_64-linux-gnu/libcudart.so.7.5.18<br>
/usr/lib/x86_64-linux-gnu/libcudnn.so.7.0</p>
<p dir="auto">If installed from source, provide</p>
<ol dir="auto">
<li>The commit hash: <a class="commit-link" data-hovercard-type="commit" data-hovercard-url="https://github.com/tensorflow/tensorflow/commit/bf31051225ce53c1c88fd45ee117a49645153770/hovercard" href="https://github.com/tensorflow/tensorflow/commit/bf31051225ce53c1c88fd45ee117a49645153770"><tt>bf31051</tt></a></li>
<li>The output of <code class="notranslate">bazel version</code><br>
Build time: Thu Jan 01 00:00:00 1970 (0)<br>
Build timestamp: Thu Jan 01 00:00:00 1970 (0)<br>
Build timestamp as int: 0</li>
</ol>
<p dir="auto">I installed cudatoolkit, and cudann following this instructions:</p>
<blockquote>
<p dir="auto">sudoapt-get install nvidia-cuda-toolkit<br>
sudo apt-get install nvidia-cuda-361-updates<br>
sudo apt-get install nvidia-nsight<br>
sudo apt-get install nvidia-profiler<br>
sudo apt-get install libcupti-dev zlib1g-dev</p>
<h1 dir="auto">Put symlinks in /usr/local/cuda</h1>
<p dir="auto">sudo mkdir /usr/local/cuda<br>
cd /usr/local/cuda<br>
sudo ln -s  /usr/lib/x86_64-linux-gnu/ lib64<br>
sudo ln -s  /usr/include/ include<br>
sudo ln -s  /usr/bin/ bin<br>
sudo ln -s  /usr/lib/x86_64-linux-gnu/ nvvm<br>
sudo mkdir -p extras/CUPTI<br>
cd extras/CUPTI<br>
sudo ln -s  /usr/lib/x86_64-linux-gnu/ lib64<br>
sudo ln -s  /usr/include/ include</p>
<h1 dir="auto">Install cudann</h1>
<h1 dir="auto"><a href="http://askubuntu.com/questions/767269/how-can-i-install-cudnn-on-ubuntu-16-04" rel="nofollow">http://askubuntu.com/questions/767269/how-can-i-install-cudnn-on-ubuntu-16-04</a></h1>
<h1 dir="auto">Download cudann as detailed above and extract</h1>
<p dir="auto">cd ~/Downloads/cuda<br>
sudo cp include/cudnn.h /usr/include<br>
sudo cp lib64/libcudnn* /usr/lib/x86_64-linux-gnu/<br>
sudo chmod a+r /usr/lib/x86_64-linux-gnu/libcudnn*</p>
<h1 dir="auto">... Install TensorFlow from source ...</h1>
</blockquote>
<p dir="auto">instruction found at  <a href="https://devtalk.nvidia.com/default/topic/936212/cuda-setup-and-installation/tensorflow-cannot-find-cudnn-ubuntu-16-04-cuda7-5-/post/4880549/#4880549" rel="nofollow">https://devtalk.nvidia.com/default/topic/936212/cuda-setup-and-installation/tensorflow-cannot-find-cudnn-ubuntu-16-04-cuda7-5-/post/4880549/#4880549</a></p>
<p dir="auto">I ecounter the problems described in issue  <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="133111577" data-permission-text="Title is private" data-url="https://github.com/tensorflow/tensorflow/issues/1066" data-hovercard-type="issue" data-hovercard-url="/tensorflow/tensorflow/issues/1066/hovercard" href="https://github.com/tensorflow/tensorflow/issues/1066">#1066</a>. I solved it using the solution described by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/chrisburr/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/chrisburr">@chrisburr</a>: <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/drufat/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/drufat">@drufat</a> I ran into a similar issue when compiling on Arch and found the solution in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="137662453" data-permission-text="Title is private" data-url="https://github.com/tensorflow/tensorflow/issues/1346" data-hovercard-type="issue" data-hovercard-url="/tensorflow/tensorflow/issues/1346/hovercard" href="https://github.com/tensorflow/tensorflow/issues/1346">#1346</a>. I'm unsure if -D__STRICT_ANSI__ is actually required but the following patch worked for me:</p>
<p dir="auto">diff --git a/third_party/gpus/crosstool/CROSSTOOL b/third_party/gpus/crosstool/CROSSTOOL<br>
index dfde7cd..15fa9fd 100644<br>
--- a/third_party/gpus/crosstool/CROSSTOOL<br>
+++ b/third_party/gpus/crosstool/CROSSTOOL<br>
@@ -46,6 +46,9 @@ toolchain {</p>
<h1 dir="auto">Use "-std=c++11" for nvcc. For consistency, force both the host compiler</h1>
<h1 dir="auto">and the device compiler to use "-std=c++11".</h1>
<p dir="auto">cxx_flag: "-std=c++11"</p>
<ul dir="auto">
<li>cxx_flag: "-D_MWAITXINTRIN_H_INCLUDED"</li>
<li>cxx_flag: "-D_FORCE_INLINES"</li>
<li>cxx_flag: "-D__STRICT_ANSI__"<br>
linker_flag: "-lstdc++"<br>
linker_flag: "-B/usr/bin/"</li>
</ul>
<p dir="auto">Next, I got the issue <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="125048279" data-permission-text="Title is private" data-url="https://github.com/tensorflow/tensorflow/issues/698" data-hovercard-type="pull_request" data-hovercard-url="/tensorflow/tensorflow/pull/698/hovercard" href="https://github.com/tensorflow/tensorflow/pull/698">#698</a>.</p>
<p dir="auto">Solved it by compiling with --genrule_strategy=standalone --spawn_strategy=standalone, as suggested by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/damienmg/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/damienmg">@damienmg</a></p>
<p dir="auto"><strong>System information</strong></p>
<ul dir="auto">
<li>TensorFlow version (you are using): 1.12</li>
<li>Are you willing to contribute it (Yes/No): Yes</li>
</ul>
<p dir="auto"><strong>Describe the feature and the current behavior/state.</strong><br>
<code class="notranslate">AttentionWrapper</code> fuses the <code class="notranslate">cell_output</code> and the <code class="notranslate">context</code> vector using an <code class="notranslate">array_ops.concat</code> operation when specifying an attention layer: <a href="https://github.com/tensorflow/tensorflow/blob/r1.12/tensorflow/contrib/seq2seq/python/ops/attention_wrapper.py#L1067">link</a></p>
<p dir="auto">In my case, this behaviour is not flexible enough, I and would like to be able to define my own function to replace the <code class="notranslate">concat</code> operation.</p>
<p dir="auto"><strong>Will this change the current api? How?</strong><br>
Similarly to <code class="notranslate">cell_input_fn</code>, I would like to have a <code class="notranslate">attention_layer_fn</code> in <code class="notranslate">AttentionWrapper</code>, which by default will perform a <code class="notranslate">concat</code>, preserving the old behaviour. This function would be passed to <code class="notranslate">_compute_attention</code>, which could be modified like this:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="if attention_layer is not None:
  attention_layer_input = attention_layer_fn(cell_output, context)
  attention = attention_layer(attention_layer_input)
else:
  attention = context"><pre class="notranslate"><code class="notranslate">if attention_layer is not None:
  attention_layer_input = attention_layer_fn(cell_output, context)
  attention = attention_layer(attention_layer_input)
else:
  attention = context
</code></pre></div>
<p dir="auto">where</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="if attention_input_fn is None:
  attention_input_fn = (
      lambda cell_output, context: array_ops.concat([cell_output, context], 1))"><pre class="notranslate"><code class="notranslate">if attention_input_fn is None:
  attention_input_fn = (
      lambda cell_output, context: array_ops.concat([cell_output, context], 1))
</code></pre></div>
<p dir="auto"><strong>Who will benefit with this feature?</strong><br>
TensorFlow users designing new attention architectures may find this feature useful.<br>
The users have to implement their own function taking <code class="notranslate">cell_output</code> and <code class="notranslate">context</code> as inputs.</p>
<p dir="auto"><strong>Any Other info.</strong><br>
I am not a super experienced developer, so I would also like to ask for advice.</p>
<p dir="auto">I implemented the feature locally and successfully trained a network, but ran into a new problem.</p>
<p dir="auto">The main goal would be to debug/visualise some internal ops defined by the user inside the new function, just like with <code class="notranslate">alignment_history</code>. And here is where things get complicated.</p>
<p dir="auto">Since these operations are created inside the <code class="notranslate">while_loop</code> of <code class="notranslate">dynamic_rnn</code>, I'd have to use a <code class="notranslate">TensorArray</code> and write the variables at every step (otherwise it says that the operation has been marked as not fetchable), but this also requires <code class="notranslate">state.time</code> when calling <code class="notranslate">write()</code>. Then the object returned by <code class="notranslate">write()</code> should be added to the <code class="notranslate">AttentionWrapperState</code>, to show up in the final_state when running <code class="notranslate">dynamic_rnn</code>. Is there a simpler way to visualise all the necessary ops inside the envisaged <code class="notranslate">attention_input_fn</code> ?</p>