<i>Using the SIMD units vs scalar units increases power consumption</i><p>True, but only slightly.  If SIMD works for the problem, the overall efficiency can be much greater. This paper has some good numbers: <a href="http:&#x2F;&#x2F;kentcz.com&#x2F;downloads&#x2F;P149-ISCA14-Preprint.pdf" rel="nofollow">http:&#x2F;&#x2F;kentcz.com&#x2F;downloads&#x2F;P149-ISCA14-Preprint.pdf</a><p><i>I wonder if this is actually a net loss.</i><p>No, on modern processors SIMD is usually a big win. The basic summary is that &quot;fixed costs dominate variable costs&quot;.  Increasing vector width (if the problem allows it) translates almost directly to increased efficiency: &quot;Doubling the SIMD width with AVX instructions doubles performance and only increases total power by 4.3%, which results in a 1.9x improvement in energy efficiency.&quot; (quotes from the paper)<p>More generally, higher Instructions Per Cycle (IPC) means higher efficiency.  The overhead of the processor is a big factor.  For computationally intense problems, working faster and finishing in fewer cycles makes for higher energy efficiency.