I’m sure there are better ways, but this should work:
tmp = torch.cat((diag[:-1].unsqueeze(1), off_diag.view(3, 4)), dim=1)
res = torch.cat((tmp.view(-1), diag[-1].unsqueeze(0))).view(4, 4)
print(res)
> tensor([[11, 12, 13, 14],
          [21, 22, 23, 24],
          [31, 32, 33, 34],
          [41, 42, 43, 44]])