I tried it on a system with 8x A100s and get a valid output:
1.8.0
True
8
A100-SXM4-40GB
/usr/local/cuda
tensor([[ 0.0440,  1.6352, -0.7515],
        [ 1.6543, -0.5374, -0.8127]], device='cuda:0')
tensor([[ 0.0440,  1.6352, -0.7515],
        [ 1.6543, -0.5374, -0.8127]], device='cuda:1')

Used driver: 460.32.03.
Could you check dmesg for any Xids and post them here?