Unfortunately, I cannot reproduce the issue on our 8x A100 node using the pip wheels, conda binaries, and a source build. Are you seeing the same behavior on all GPUs (could you try to use another one, as you are currently using GPU1)? Did this behavior start after a specific update (of the driver etc.)?