You could merge the indexing or use view instead:
t=torch.tensor([[1,2],[3,4]])
r=torch.randperm(2)
c=torch.randperm(2)
t=t[r[:, None], c]

# With view
idx = torch.randperm(t.nelement())
t = t.view(-1)[idx].view(t.size())