This is not really about pytorch/autograd as such. Although I do use pytorch And I do sneak in a quick reference to autograd at the end  . Anhyway, sneaking these in here  . These basically give an intro to the maths of backprop in an rnn setting, going into chain rule, derivatives etc, and then writing it all out in python. Finally, do the same thing using autograd, which just takes one line 
part 1: introd, forward prop, including python code
backprop rnn part1 forwardprop concepts and python
part 2: backprop concepts
backprop rnn part2 backprop concepts
part 3: why backprop?
backprop rnn part3 backprop why
part 4: backprop maths. chainrule
backprop rnn part4 backprop maths chainrule
part 5: definitions of gradOutput and gradWEights, for python code
backprop rnn part5 definitions
part 6: python code
backprop rnn part6 backprop python
part 7: using autograd instead. one line 
backprop rnn part7 autograd