Yes, you should still disable the gradient calculation using one of the mentioned approaches.
If your training consists of two steps, I guess you could use two different training methods or just add the second step after the first training run is done.


This error is raised, as the cudnn RNN implementation cannot calculate gradients during the evaluation. Please refer to the previous posts for more information.