One observation: allocation in cuda is a sync point, which will be slow. If you can somehow create the permuted variable once, before any loop, and simply copy the permuted values across, without allocation, each iteration, that might speed things up?