Yes this is ok to do. The Tensor and its gradient doesn’t need to have the same contiguity.

is calling tensor.contiguous() blocking?

What do you mean by blocking?
It is a differentiable op, so gradients will flow back through it as expected.