I have a very similar problem, in my case I use an RNN and then I get
"CudnnRNNLegacyBackward is not differentiable twice"
I would modify things myself just to make it be differentiable the way I want (if needed). How can I overcome this?
Any help would be awesome. I use GPU, Cuda 8, and did not compile from scratch. (I do not use nn.parallel)
I appreciate any help!