<h3 dir="auto">System information</h3>
<ul dir="auto">
<li><strong>Have I written custom code (as opposed to using a stock example script provided in TensorFlow)</strong>: N/A</li>
<li><strong>OS Platform and Distribution (e.g., Linux Ubuntu 16.04)</strong>: N/A</li>
<li><strong>Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device</strong>: N/A</li>
<li><strong>TensorFlow installed from (source or binary)</strong>: N/A</li>
<li><strong>TensorFlow version (use command below)</strong>: N/A</li>
<li><strong>Python version</strong>: N/A</li>
<li><strong>Bazel version (if compiling from source)</strong>: N/A</li>
<li><strong>GCC/Compiler version (if compiling from source)</strong>: N/A</li>
<li><strong>CUDA/cuDNN version</strong>: N/A</li>
<li><strong>GPU model and memory</strong>: N/A</li>
<li><strong>Exact command to reproduce</strong>: N/A</li>
</ul>
<h3 dir="auto">Describe the problem</h3>
<p dir="auto">Currently, the TensorFlow Python library is released in two flavours: CPU-only and CPU/GPU. The latter is dynamically linked with CUDA libraries and fails to load if they are not available in the linker path.</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="ImportError: dlopen([...]/tensorflow/python/_pywrap_tensorflow_internal.so, 10): Library not loaded: @rpath/libcublas.8.0.dylib
  Referenced from: [...]/tensorflow/python/_pywrap_tensorflow_internal.so
  Reason: image not found"><pre class="notranslate"><span class="pl-v">ImportError</span>: <span class="pl-en">dlopen</span>([...]<span class="pl-c1">/</span><span class="pl-s1">tensorflow</span><span class="pl-c1">/</span><span class="pl-s1">python</span><span class="pl-c1">/</span><span class="pl-s1">_pywrap_tensorflow_internal</span>.<span class="pl-s1">so</span>, <span class="pl-c1">10</span>): <span class="pl-v">Library</span> <span class="pl-c1">not</span> <span class="pl-s1">loaded</span>: @<span class="pl-s1">rpath</span><span class="pl-c1">/</span><span class="pl-s1">libcublas</span>.<span class="pl-c1">8.0</span>.<span class="pl-s1">dylib</span>
  <span class="pl-v">Referenced</span> <span class="pl-s1">from</span>: [...]<span class="pl-c1">/</span><span class="pl-s1">tensorflow</span><span class="pl-c1">/</span><span class="pl-s1">python</span><span class="pl-c1">/</span><span class="pl-s1">_pywrap_tensorflow_internal</span>.<span class="pl-s1">so</span>
  <span class="pl-v">Reason</span>: <span class="pl-s1">image</span> <span class="pl-c1">not</span> <span class="pl-s1">found</span></pre></div>
<p dir="auto">I suspect that it might be possible to load CUDA libraries dynamically at runtime. This would allow having a single TensorFlow build which attempts to load CUDA, and if it is not available, falls back to CPU-only ops.</p>
<p dir="auto">It hard for me to estimate the feasibility of the proposal for the current TensorFlow runtime implementation, so it might as well be the case, that the proposed change is too big/intrusive to be practical.</p>
<p dir="auto">I did not find a PR for this <a href="https://github.com/tensorflow/tensorflow/commit/cbfd50ff0f01e1825922230a8bc6e5766da98dd7">patch</a>. Maybe this discussion belongs to that particular commit page. If so, I will move it there.</p>
<p dir="auto">Can somebody explain the rationale behind this patch? Current contrib/verbs uses exactly this flag to transfer tensors. Note it is set true <a href="https://github.com/tensorflow/tensorflow/blob/r1.3/tensorflow/contrib/verbs/rdma_rendezvous_mgr.cc#L34">here</a>. Removing it totally breaks verbs. I am asking if this patch is absolutely needed since there is no simple workaround for contrib/verbs without this flag.</p>