Hey Simon,
I’ve run your code, and it’s not failing for me. Also it always computes 0.0. Is this unexpected:
CUDA_VISIBLE_DEVICES="1,3" python test.py
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
switching to DataParallel mode
for me, this works with
    export CUDA_VISIBLE_DEVICES="0"
    export CUDA_VISIBLE_DEVICES="1"
    export CUDA_VISIBLE_DEVICES="2"
    export CUDA_VISIBLE_DEVICES="3"
    export CUDA_VISIBLE_DEVICES="0,1"
    export CUDA_VISIBLE_DEVICES="2,3"
and fails with many others like
    export CUDA_VISIBLE_DEVICES="0,2"
    export CUDA_VISIBLE_DEVICES="0,3"
    export CUDA_VISIBLE_DEVICES="1,2"
    export CUDA_VISIBLE_DEVICES="1,3"
    export CUDA_VISIBLE_DEVICES="0,1,2,3"
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0