You can use the same trick described in tensorflow fill_triangular, but converted to PyTorch, here is an example for the lower triangular of a 3x3 matrix:
x = torch.arange(6) + 1 
xc = torch.cat([x[3:], x.flip(dims=[0])])
y = xc.view(3, 3)
torch.tril(y)

tensor([[4, 0, 0],
        [6, 5, 0], 
        [3, 2, 1]])