<p dir="auto">I haven't been able to find anything online that documents how the sync of reusable variables happens between CPU and GPU. Specifically, I'm interested in if a CPU pinned variable is used multiple times in a GPU computation without being updated, does it sync only once (i.e. use a dirty flag), or will it try sync again each time?</p>
<p dir="auto">Hello dear tensorflowers,</p>
<p dir="auto">In a research project, I encountered the need to create tensors of the same shape than any other tensor with a custom value (not just 0 or 1), could be Boolean, Floats, Integers and so on.</p>
<p dir="auto">The functions prototypes are the following and will be implemented in <a href="https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/ops/array_ops.py"> tensorflow/python/ops/array_ops.py</a>:</p>
<ul dir="auto">
<li><strong>tf.alphas</strong> (shape, alpha_value, name=None)</li>
<li><strong>tf.alphas_like</strong> (tensor, alpha_value, name=None, optimize=True)</li>
</ul>
<p dir="auto">The code is not created from scratch. It is <strong>highly</strong> inspired by the functions tf.ones, tf.zeros for tf.alphas and by tf.zeros_like, tf.ones_like for tf.alphas_like.</p>
<p dir="auto">The code use the latest implementation and has been designed to work with <em>eager_mode</em>.<br>
The number of modification is relatively small, thus I am relatively confident on the robustness of the new implementation (largely based on the existing one).</p>
<p dir="auto">The idea is to reproduce and merge the functions while enabling to set any custom value in the tensor:</p>
<ul dir="auto">
<li><strong>tf.alphas merges:</strong>
<ul dir="auto">
<li>tf.ones</li>
<li>tf.zeros</li>
</ul>
</li>
<li><strong>tf.alphas_like merges:</strong>
<ul dir="auto">
<li>tf.zeros_like</li>
<li>tf.ones_like</li>
</ul>
</li>
</ul>
<h3 dir="auto">How is the API Working ?</h3>
<p dir="auto">My new functions take a parameter <em>alpha_value</em> and fill the tensor with this value. This allows me to run such a script:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="import tensorflow as tf

a = tf.constant([
    [
        [4, 5, 6],
        [1, 2, 3]
    ],
    [
        [4, 5, 6],
        [1, 2, 3]
    ]
])

b1 = tf.alphas_like(a, 0.5431)
b2 = tf.alphas_like(a, 5)
b3 = tf.alphas_like(a, -5)
b4 = tf.alphas_like(a, True)

with tf.Session() as sess:
    _b1, _b2, _b3, _b4 = sess.run([b1, b2, b3, b4])
    
print(&quot;b1:&quot;, _b1)
print(&quot;b2:&quot;, _b2)
print(&quot;b3:&quot;, _b3)
print(&quot;b4:&quot;, _b4)

############### OUTPUTS ###############

&gt;&gt;&gt; b1: [
  [
    [ 0.5431  0.5431  0.5431]
    [ 0.5431  0.5431  0.5431]
  ]
  [
    [ 0.5431  0.5431  0.5431]
    [ 0.5431  0.5431  0.5431]
  ]
]

&gt;&gt;&gt; b2: [
  [
    [5 5 5]
    [5 5 5]
  ]
  [
    [5 5 5]
    [5 5 5]
  ]
]

&gt;&gt;&gt; b3: [
  [
    [-5 -5 -5]
    [-5 -5 -5]
  ]
  [
    [-5 -5 -5]
    [-5 -5 -5]
  ]
]

&gt;&gt;&gt; b4: [
  [
    [ True  True  True]
    [ True  True  True]
  ]
  [
    [ True  True  True]
    [ True  True  True]
  ]
]"><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-s1">tensorflow</span> <span class="pl-k">as</span> <span class="pl-s1">tf</span>

<span class="pl-s1">a</span> <span class="pl-c1">=</span> <span class="pl-s1">tf</span>.<span class="pl-en">constant</span>([
    [
        [<span class="pl-c1">4</span>, <span class="pl-c1">5</span>, <span class="pl-c1">6</span>],
        [<span class="pl-c1">1</span>, <span class="pl-c1">2</span>, <span class="pl-c1">3</span>]
    ],
    [
        [<span class="pl-c1">4</span>, <span class="pl-c1">5</span>, <span class="pl-c1">6</span>],
        [<span class="pl-c1">1</span>, <span class="pl-c1">2</span>, <span class="pl-c1">3</span>]
    ]
])

<span class="pl-s1">b1</span> <span class="pl-c1">=</span> <span class="pl-s1">tf</span>.<span class="pl-en">alphas_like</span>(<span class="pl-s1">a</span>, <span class="pl-c1">0.5431</span>)
<span class="pl-s1">b2</span> <span class="pl-c1">=</span> <span class="pl-s1">tf</span>.<span class="pl-en">alphas_like</span>(<span class="pl-s1">a</span>, <span class="pl-c1">5</span>)
<span class="pl-s1">b3</span> <span class="pl-c1">=</span> <span class="pl-s1">tf</span>.<span class="pl-en">alphas_like</span>(<span class="pl-s1">a</span>, <span class="pl-c1">-</span><span class="pl-c1">5</span>)
<span class="pl-s1">b4</span> <span class="pl-c1">=</span> <span class="pl-s1">tf</span>.<span class="pl-en">alphas_like</span>(<span class="pl-s1">a</span>, <span class="pl-c1">True</span>)

<span class="pl-k">with</span> <span class="pl-s1">tf</span>.<span class="pl-v">Session</span>() <span class="pl-k">as</span> <span class="pl-s1">sess</span>:
    <span class="pl-s1">_b1</span>, <span class="pl-s1">_b2</span>, <span class="pl-s1">_b3</span>, <span class="pl-s1">_b4</span> <span class="pl-c1">=</span> <span class="pl-s1">sess</span>.<span class="pl-en">run</span>([<span class="pl-s1">b1</span>, <span class="pl-s1">b2</span>, <span class="pl-s1">b3</span>, <span class="pl-s1">b4</span>])
    
<span class="pl-en">print</span>(<span class="pl-s">"b1:"</span>, <span class="pl-s1">_b1</span>)
<span class="pl-en">print</span>(<span class="pl-s">"b2:"</span>, <span class="pl-s1">_b2</span>)
<span class="pl-en">print</span>(<span class="pl-s">"b3:"</span>, <span class="pl-s1">_b3</span>)
<span class="pl-en">print</span>(<span class="pl-s">"b4:"</span>, <span class="pl-s1">_b4</span>)

<span class="pl-c">############### OUTPUTS ###############</span>

<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-s1">b1</span>: [
  [
    [ <span class="pl-c1">0.5431</span>  <span class="pl-c1">0.5431</span>  <span class="pl-c1">0.5431</span>]
    [ <span class="pl-c1">0.5431</span>  <span class="pl-c1">0.5431</span>  <span class="pl-c1">0.5431</span>]
  ]
  [
    [ <span class="pl-c1">0.5431</span>  <span class="pl-c1">0.5431</span>  <span class="pl-c1">0.5431</span>]
    [ <span class="pl-c1">0.5431</span>  <span class="pl-c1">0.5431</span>  <span class="pl-c1">0.5431</span>]
  ]
]

<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-s1">b2</span>: [
  [
    [<span class="pl-c1">5</span> <span class="pl-c1">5</span> <span class="pl-c1">5</span>]
    [<span class="pl-c1">5</span> <span class="pl-c1">5</span> <span class="pl-c1">5</span>]
  ]
  [
    [<span class="pl-c1">5</span> <span class="pl-c1">5</span> <span class="pl-c1">5</span>]
    [<span class="pl-c1">5</span> <span class="pl-c1">5</span> <span class="pl-c1">5</span>]
  ]
]

<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-s1">b3</span>: [
  [
    [<span class="pl-c1">-</span><span class="pl-c1">5</span> <span class="pl-c1">-</span><span class="pl-c1">5</span> <span class="pl-c1">-</span><span class="pl-c1">5</span>]
    [<span class="pl-c1">-</span><span class="pl-c1">5</span> <span class="pl-c1">-</span><span class="pl-c1">5</span> <span class="pl-c1">-</span><span class="pl-c1">5</span>]
  ]
  [
    [<span class="pl-c1">-</span><span class="pl-c1">5</span> <span class="pl-c1">-</span><span class="pl-c1">5</span> <span class="pl-c1">-</span><span class="pl-c1">5</span>]
    [<span class="pl-c1">-</span><span class="pl-c1">5</span> <span class="pl-c1">-</span><span class="pl-c1">5</span> <span class="pl-c1">-</span><span class="pl-c1">5</span>]
  ]
]

<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-s1">b4</span>: [
  [
    [ <span class="pl-c1">True</span>  <span class="pl-v">True</span>  <span class="pl-c1">True</span>]
    [ <span class="pl-c1">True</span>  <span class="pl-v">True</span>  <span class="pl-c1">True</span>]
  ]
  [
    [ <span class="pl-c1">True</span>  <span class="pl-v">True</span>  <span class="pl-c1">True</span>]
    [ <span class="pl-c1">True</span>  <span class="pl-v">True</span>  <span class="pl-c1">True</span>]
  ]
]</pre></div>
<h3 dir="auto">How can you help ?</h3>
<p dir="auto">Before submitting a PR, I would like to know a few things:</p>
<ul dir="auto">
<li>Is it something that would be any kind of interest and worth a PR?</li>
<li>Are the names I have chosen (tf.alphas and tf.alphas_like) okay with everyone ?</li>
<li>In my PR, should I delete the implementation of tf.zeros_likes and tf.ones_likes and replace them as an alias of my new function which basically does the same job, just in a more flexible way ?</li>
</ul>
<p dir="auto">Thanks for your time and attention,</p>
<p dir="auto">Best Regards,</p>
<p dir="auto">Jonathan</p>