Solved by jel158 in post #5 
              
              
                In my case, I use this simple code to see values of gradient, particularly for rnn: 
for p,n in zip(rnn.parameters(),rnn._all_weights[0]):
    if n[:6] == 'weight':
        print('===========\ngradient:{}\n----------\n{}'.format(n,p.grad))