I cannot reproduce this issue on a recent source build using CUDA11.2 and an internal cudnn version as well as the PyTorch 1.7.1 conda binaries with CUDA11.0 and get always:
tensor([0.0109, 0.0109, 0.0109, 0.0109, 0.0109, 0.0109, 0.0109, 0.0109, 0.0109,
        0.0109, 0.0109, 0.0109, 0.0109, 0.0108, 0.0108, 0.0109],

on a node with P100-SXM2 16GB GPUs.
EDIT: could you store the state_dict as well as the input from the CUDA10.2 run, load it in the CUDA11.0 environment, and check the outputs again?