Solved by tom in post #4 
              
              
                On the GPU it has its own cache, I think on mobile, too. On the CPU it uses posix_memalign which relies on libc to do caching (I think they do in contemporary systems), so I would not worry about it. 
You can also reserve the memory once and use matmul_out if you love manual, but very likely it will…