Memory-Level Parallelism: Intel Skylake versus Apple A12/A12X
Okay, I&#x27;ve given it some thought. Here&#x27;s my theory:<p>ARM&#x27;s load&#x2F;load, load&#x2F;store, store&#x2F;load, and store&#x2F;store reordering is more flexible than Intel&#x27;s store&#x2F;load (only) reordering. As such, ARM is more likely to benefit from deeper load-buffering than Intel.<p>Remember that all of these systems are multi-core, which means we have to consider what these load-buffers do during a multi-core situation. Here&#x27;s some example code for &quot;Thread 1&quot;<p>Thread 1:<p><pre><code>    register RAX=0; &#x2F;&#x2F; Non-memory. No load&#x2F;store associated
    int a=0, b=0, c=0; &#x2F;&#x2F; &quot;True Memory&quot;, accessing these have a load&#x2F;store
    while(1){
        RAX++;
        memory barrier();
        a = RAX;
        RAX++;
        memory barrier();
        b = RAX;
        RAX++;
        memory barrier();
        c = RAX;
    }
</code></pre>
In effect, this thread would create the memory commands:<p><pre><code>    memory barrier
    store [A], 0
    memory barrier
    store [B], 1
    memory barrier
    store [C], 2
    memory barrier
    store [A], 3
    memory barrier
    store [B], 4
    memory barrier
    store [C], 5
    (etc. etc)
</code></pre>
For simplicity, I&#x27;ve added memory barriers to ensure that these writes happen in EXACTLY this order. No need to worry about ARM&#x27;s store&#x2F;store reordering in this example. (x86 does NOT need memory barriers, because store&#x2F;store will always happen in order)<p>Now lets imagine a 2nd core reading from these variables. Lets say Core#2 has the following load buffer:<p><pre><code>    load rax, [A]
    load rbx, [B]
    load rcx, [C]
    # Note: no memory barriers!
</code></pre>
With this ordering, the Intel processor is guaranteed to have &quot;RBX&quot; a later value than RAX, and RCX to have the latest value.<p>This is NOT true in ARM. In the ARM processor, the following assignments are possible:<p><pre><code>    RAX = 3 # load RAX MAY execute last on ARM.
    RBX = 1 # This example will NEVER happen on x86
    RCX = 2 
</code></pre>
Why? Because in ARM, the load&#x2F;load boundary can be crossed by the core. That is, the &quot;load RBX, [B]&quot; instruction MAY be executed first on ARM&#x27;s architecture, despite coming in afterwards.<p>This means that load buffers are far, far cheaper on ARM. But they also come at a cost to the programmer&#x27;s sanity. If you actually have important &quot;ordering&quot;, you need to be sure to protect your code with memory barriers.<p>x86 has an implicit memory barrier on all stores and loads. This is slower and more costly. In effect, the load-buffer on x86 MUST be a proper queue: the first loads must execute first, while the last loads must execute last.<p>In ARM, this isn&#x27;t the case. A load-buffer of size 40 can all execute simultaneously and out of order. Much to the confusion to multi-threaded programmers, I&#x27;m sure. This example also demonstrates why on weakly ordered systems, you need a memory-barrier on BOTH threads. (Fixing the store&#x2F;store reordering with memory barriers only solves half the problem)<p>With that being said: modern programmers know that the only variables that need a defined ordering are synchronization structures, like spinlocks. Every spinlock needs a &quot;release&quot; barrier afterwards (Explicit barrier states: do NOT reorder anything past this store). Every spinlock needs an &quot;acquire&quot; barrier before reading (Command: do NOT reorder anything before this load).<p>Semaphores &#x2F; Atomics &#x2F; other synchronization structures also need a defined ordering. Accessing &quot;normal&quot; variables is well known to be &quot;unordered&quot; by modern programmers. So ARM has the advantage with regards to modern languages. x86&#x27;s architecture is more strict than necessary.