Thanks.
I have checked the performance with and without cuDNN for RNN. It seems that with cuDNN, RNN can run much faster about 10 times. Is this really the case?
I have also implemented the same code of RNN1 in Lasagne based on Theano. It seems working in a similar speed as RNN in pytorch with cuDNN.
What is the difference in using cuDNN between theano and pytorch? Is is automatically used in theano?