I think you are  right.
When I run my codes, I found this error happened again and when I disabled cuDNN, the error disapeared. I think cuDNN may not so flexible for processing data in all shapes. That may be a bug in cuDNN. And as I posted, I found that sometimes using smaller batch_size can avoid this error. However, in these cases, the occupation percentage of GPU memory is very low.
My environment: cuda v10.0.130, cudnn v7.6.5, V100.