It could be a known issue with e.g. older cuDNN versions.
Since you are using the CUDA 10.2 binaries, your cuDNN version would be 7.6.5, so could you update to the latest CUDA11 binaries?
With CUDA 11.5 and cuDNN 8.3.2 I get:
1.11.0.dev20220108+cu115
11.5
NVIDIA GeForce RTX 3090
kernel: 1 , stride: 1 , H: 1 , W: 1
all nans: True   all infs: False
kernel: 3 , stride: 1 , H: 3 , W: 74
all nans: True   all infs: False
kernel: 3 , stride: 1 , H: 3 , W: 75
all nans: True   all infs: False
kernel: 3 , stride: 1 , H: 2048 , W: 74
all nans: True   all infs: False
kernel: 3 , stride: 1 , H: 2048 , W: 75
all nans: True   all infs: False
kernel: 1 , stride: 2 , H: 1 , W: 1
all nans: True   all infs: False
kernel: 1 , stride: 2 , H: 2 , W: 1
all nans: True   all infs: False
kernel: 1 , stride: 2 , H: 1 , W: 2048
all nans: True   all infs: False
kernel: 1 , stride: 2 , H: 2 , W: 2048
all nans: True   all infs: False
kernel: 3 , stride: 2 , H: 10 , W: 3
all nans: True   all infs: False
kernel: 3 , stride: 2 , H: 11 , W: 3
all nans: True   all infs: False
kernel: 3 , stride: 2 , H: 3 , W: 14
all nans: True   all infs: False
kernel: 3 , stride: 2 , H: 3 , W: 15
all nans: True   all infs: False