title: How to merge a scalar into a vector without the compiler wasting an instruction zeroing upper elements? Design limitation in Intel's intrinsics?
body: <p>I don't have a particular use-case in mind; I'm asking if this is really a design flaw / limitation in Intel's intrinsics or if I'm just missing something.</p>
  <p>If you want to combine a scalar float with an existing vector, there doesn't seem to be a way to do it without high-element-zeroing or broadcasting the scalar into a vector, using Intel intrinsics.  I haven't investigated GNU C native vector extensions and the associated builtins.</p>
  <p>This wouldn't be too bad if the extra intrinsic optimized away, but it doesn't with gcc (5.4 or 6.2).  There's also no nice way to use <code>pmovzx</code> or <code>insertps</code> as loads, for the related reason that their intrinsics only take vector args.  (And gcc doesn't fold a scalar-&gt;vector load into the asm instruction.)</p>
  <pre><code>__m128 replace_lower_two_elements(__m128 v, float x) {
    __m128 xv = _mm_set_ss(x);        // WANTED: something else for this step, some compilers actually compile this to a separate insn
    return _mm_shuffle_ps(v, xv, 0);  // lower 2 elements are both x, and the garbage is gone
  }
  </code></pre>
  <p>gcc 5.3 -march=nehalem -O3 output, to enable SSE4.1 and tune for that Intel CPU:  (It's even worse without SSE4.1; multiple instructions to zero the upper elements).</p>
  <pre><code>    insertps  xmm1, xmm1, 0xe    # pointless zeroing of upper elements.  shufps only reads the low element of xmm1
      shufps    xmm0, xmm1, 0      # The function *should* just compile to this.
      ret
  </code></pre>
  <p><strong>TL:DR: the rest of this question is just asking if you can actually do this efficiently, and if not why not.</strong></p>
  <hr />
  <p>clang's shuffle-optimizer gets this right, and doesn't waste instructions on zeroing high elements (<code>_mm_set_ss(x)</code>), or duplicating the scalar into them (<code>_mm_set1_ps(x)</code>).  Instead of writing something the compiler has to optimize away, shouldn't there be a way to write it &quot;efficiently&quot; in C in the first place?  Even very recent gcc <em>doesn't</em> optimize it away, so this is a real (but minor) problem.</p>
  <hr />
  <p>This would be possible if there was a scalar-&gt;128b equivalent of <a href="https://software.intel.com/sites/landingpage/IntrinsicsGuide/#expand=3050,4590,3544,577,577&amp;text=castps128" rel="nofollow noreferrer"><code>__m256 _mm256_castps128_ps256 (__m128 a)</code></a>.  i.e. produce a <code>__m128</code> with undefined garbage in upper elements, and the float in the low element, compiling to zero asm instructions if the scalar float/double was already in an xmm register.</p>
  <p><strong>None of the following intrinsics exist, but they should</strong>.</p>
  <ul>
  <li><p>a scalar-&gt;__m128 equivalent of <code>_mm256_castps128_ps256</code> as described above.  The most general solution for the scalar-already-in-register case.</p>
  </li>
  <li><p><code>__m128 _mm_move_ss_scalar (__m128 a, float s)</code>: replace low element of vector <code>a</code> with scalar <code>s</code>.  This isn't actually necessary if there's a general-purpose scalar-&gt;__m128 (previous bullet point).  (The reg-reg form of <a href="http://www.felixcloutier.com/x86/MOVSS.html" rel="nofollow noreferrer"><code>movss</code></a> merges, unlike the load form which zeros, and unlike <a href="http://www.felixcloutier.com/x86/MOVD:MOVQ.html" rel="nofollow noreferrer"><code>movd</code></a> which zeros upper elements in both cases.  To copy a register holding a scalar float without false dependencies, use <a href="http://www.felixcloutier.com/x86/MOVAPS.html" rel="nofollow noreferrer"><code>movaps</code></a>).</p>
  </li>
  <li><p><code>__m128i _mm_loadzxbd (const uint8_t *four_bytes)</code> and other sizes of <a href="http://www.felixcloutier.com/x86/PMOVZX.html" rel="nofollow noreferrer">PMOVZX</a> / PMOVSX:  <a href="https://stackoverflow.com/questions/34279513/loading-8-chars-from-memory-into-an-m256-variable-as-packed-single-precision-f">AFAICT, there's no good safe way to use the PMOVZX intrinsics as a load</a>, because the inconvenient safe way doesn't optimize away with gcc.</p>
  </li>
  <li><p><code>__m128 _mm_insertload_ps (__m128 a, float *s, const int imm8)</code>.   <a href="http://www.felixcloutier.com/x86/INSERTPS.html" rel="nofollow noreferrer">INSERTPS</a> behaves differently as a load: the upper 2 bits of the imm8 are ignored, and it always takes the scalar at the effective address (instead of an element from a vector in memory).  This lets it work with addresses that aren't 16B-aligned, and work even without faulting if the <code>float</code> right before an unmapped page.</p>
  <p>Like with PMOVZX, gcc fails to fold an upper-element-zeroing <code>_mm_load_ss()</code> into a memory operand for INSERTPS.  (Note that if the upper 2 bits of the imm8 aren't both zero, then <code>_mm_insert_ps(xmm0, _mm_load_ss(), imm8)</code> can compile to <code>insertps xmm0,xmm0,foo</code>, with a different imm8 that zeros elements in vec as-if the src element was actually a zero produced by MOVSS from memory.  Clang actually uses XORPS/BLENDPS in that case)</p>
  </li>
  </ul>
  <hr />
  <p><strong>Are there any viable workarounds</strong> to emulate any of those that are both safe (don't break at -O0 by e.g. loading 16B that might touch the next page and segfault), and efficient (no wasted instructions at -O3 with current gcc and clang at least, preferably also other major compilers)?  Preferably also in a readable way, but if necessary it could be put behind an inline wrapper function like <code>__m128 float_to_vec(float a){ something(a); }</code>.</p>
  <p>Is there any good reason for Intel not to introduce intrinsics like that?  They could have added a float-&gt;__m128 with undefined upper elements at the same time as adding <code>_mm256_castps128_ps256</code>.  <strong>Is this a matter of compiler internals making it hard to implement?</strong>  Perhaps specifically ICC internals?</p>
  <hr />
  <p>The major calling conventions on x86-64 (SysV or MS <code>__vectorcall</code>) take the first FP arg in xmm0 and return scalar FP args in xmm0, with upper elements undefined.  (See the  <a href="/questions/tagged/x86" class="post-tag" title="show questions tagged &#39;x86&#39;" rel="tag">x86</a> tag wiki for ABI docs).  This means it's not uncommon for the compiler to have a scalar float/double in a register with unknown upper elements.  This will be rare in a vectorized inner loop, so I think avoiding these useless instructions will mostly just save a bit of code size.</p>
  <p>The pmovzx case is more serious: that is something you might use in an inner loop (e.g. for a LUT of VPERMD shuffle masks, saving a factor of 4 in cache footprint vs. storing each index padded to 32 bits in memory).</p>
  <hr />
  <p>The pmovzx-as-a-load issue has been bothering me for a while now, and <a href="https://stackoverflow.com/questions/38963947/are-there-move-mm-move-ss-and-set-mm-set-ss-intrinsics-that-work-for-doubl">the original version of this question</a> got me thinking about the related issue of using a scalar float in an xmm register.  There are probably more use-cases for pmovzx as a load than for scalar-&gt;__m128.</p>