What I mean by “being contiguous” is that the values in the tensor lies in a contiguous chunk of memory according to the the order of dimensions (so first dim has largest stride, and last dim has stride 1, etc.). E.g., torch.randn(1,5).t() is still contiguous, but torch.randn(2,5).t() is not.
I suggest just writing it with vectorized operations. Your current code will be very slow anyways.