Solved by jpeg729 in post #3 
              
              
                I have never had to call .contiguous() after .permute(). 
As I understand it, permute works by changing the strides of the view mechanism. The data isn’t moved at all so if it was contiguous before permuting, then it still is. 
The only caveat is if you slice the tensor after permuting the dimension…