<p dir="auto">'Training Deep Nets with Sublinear Memory Cost' and 'Memory-Efficient Implementation of DenseNets' indicate that use drop intermediate feature map and recompute it if needed can save memory(while add computation burden),</p>
<p dir="auto">so we need some mechanism to annotate some op's inputs  <code class="notranslate">recomputable</code> , and drop this input memory after op finish(set input's reference count to 0), when need this op again, recompute it.</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="a = tf.get_varible(shape=[None,10])
b = tf.get_varible(shape=[None,10])
c = tf.get_varible(shape=[None,10])
d = a*b
d_recomputable = tf.recomputable(d)
e = d_recomputable+c"><pre class="notranslate"><span class="pl-s1">a</span> <span class="pl-c1">=</span> <span class="pl-s1">tf</span>.<span class="pl-en">get_varible</span>(<span class="pl-s1">shape</span><span class="pl-c1">=</span>[<span class="pl-c1">None</span>,<span class="pl-c1">10</span>])
<span class="pl-s1">b</span> <span class="pl-c1">=</span> <span class="pl-s1">tf</span>.<span class="pl-en">get_varible</span>(<span class="pl-s1">shape</span><span class="pl-c1">=</span>[<span class="pl-c1">None</span>,<span class="pl-c1">10</span>])
<span class="pl-s1">c</span> <span class="pl-c1">=</span> <span class="pl-s1">tf</span>.<span class="pl-en">get_varible</span>(<span class="pl-s1">shape</span><span class="pl-c1">=</span>[<span class="pl-c1">None</span>,<span class="pl-c1">10</span>])
<span class="pl-s1">d</span> <span class="pl-c1">=</span> <span class="pl-s1">a</span><span class="pl-c1">*</span><span class="pl-s1">b</span>
<span class="pl-s1">d_recomputable</span> <span class="pl-c1">=</span> <span class="pl-s1">tf</span>.<span class="pl-en">recomputable</span>(<span class="pl-s1">d</span>)
<span class="pl-s1">e</span> <span class="pl-c1">=</span> <span class="pl-s1">d_recomputable</span><span class="pl-c1">+</span><span class="pl-s1">c</span></pre></div>
<p dir="auto">relate issue</p>
<ul dir="auto">
<li><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="148266318" data-permission-text="Title is private" data-url="https://github.com/tensorflow/tensorflow/issues/1934" data-hovercard-type="issue" data-hovercard-url="/tensorflow/tensorflow/issues/1934/hovercard" href="https://github.com/tensorflow/tensorflow/issues/1934">#1934</a></li>
<li><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="256532521" data-permission-text="Title is private" data-url="https://github.com/tensorflow/tensorflow/issues/12948" data-hovercard-type="issue" data-hovercard-url="/tensorflow/tensorflow/issues/12948/hovercard" href="https://github.com/tensorflow/tensorflow/issues/12948">#12948</a></li>
</ul>
<p dir="auto">in short, normal reference add reference count, recomputable reference do not add reference count</p>
<p dir="auto">The python 3 gpu package listed on: <a href="https://www.tensorflow.org/install/install_mac#the_url_of_the_tensorflow_python_package" rel="nofollow">https://www.tensorflow.org/install/install_mac#the_url_of_the_tensorflow_python_package</a> is missing.</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="$ curl https://storage.googleapis.com/tensorflow/mac/gpu/tensorflow_gpu-1.1.0-py3-none-any.whl
&lt;?xml version='1.0' encoding='UTF-8'?&gt;&lt;Error&gt;&lt;Code&gt;NoSuchKey&lt;/Code&gt;&lt;Message&gt;The specified key does not exist.&lt;/Message&gt;&lt;/Error&gt;"><pre class="notranslate"><code class="notranslate">$ curl https://storage.googleapis.com/tensorflow/mac/gpu/tensorflow_gpu-1.1.0-py3-none-any.whl
&lt;?xml version='1.0' encoding='UTF-8'?&gt;&lt;Error&gt;&lt;Code&gt;NoSuchKey&lt;/Code&gt;&lt;Message&gt;The specified key does not exist.&lt;/Message&gt;&lt;/Error&gt;
</code></pre></div>