Interesting, does it have to be inplace?
How about creating an inverted identity diagonal matrix, and then multiply element wise?
your_tensor *= (1 - torch.eye(N, N))
Also checkout fill_diagonal_ 83 function it might fit perfectly
Roy.