I am also facing this same issue. I did not face it when I was using a small dataset (by small here I mean the number of features). When I ran the same code on a larger dataset (larger in number of features), I had to increase the size of parameters of the RNN and CNN layers I am using. It is then when I got this error.
Using torch.backends.cudnn.enabled = False resolves this, but the execution becomes very slow.
I am not able to understand how to resolve this error and why is this “workaround” working?