title: How to manage hardware cache usage
body: <p>I wonder if there is a way to directly manage which parts of the memory are loaded into (<em>and kept in</em>) the cache. I know that there is __builtin_prefetch() in gcc, but I would need a bit more (at least I think I would...). </p>
  
  <p>Here is my problem:
  I have a loop where I update a vector v (<em>very</em> pseudocode):</p>
  
  <pre><code>while (1) {
      update_an_entry_in_v;
  }
  </code></pre>
  
  <p>The length of v is in the millions. The loop runs 100s of millions times, and it jumps around in v. Therefore my cache-hit ratio is really bad, and there is no way of getting all of v into L1 cache.</p>
  
  <p>I can reorganize the code like this:</p>
  
  <pre><code>for (i=0; i &lt; length_of_v; i+=1024) {
      while (1) {
          update_an_entry_in_v[i:i+1023];
      }
  }
  </code></pre>
  
  <p>That is, first only work with those updates that I know will update only entries in v[0:1023], process all of those, then go to the next block, etc. </p>
  
  <p>Now this is probably going to have a much better cache behaviour, but I wonder if it is possible to make it even better. Let's focus on the v[0:1023] block. I still will jump around within v[0:1023], so the hardware may think that there is no need to keep parts of that chunk of memory in L1 cache any longer, and then, boom!, I go back to that chunk again.</p>
  
  <p>So the question becomes: is there a way to mark the whole v[0:1023] so that it gets loaded into L1 before the inner while loop, and then "unmark" it once the while loop is finished?</p>
  
  <p>(Obviously, 1024 is just a convenient number that should be tuned. With doubles that's 8K memory, and given that these days most machines have at least 32-64K L1 data cache that should leave enough for everything else.)</p>