Actually my notebook is modified version of that nice notebook 2 in which there’s no optimizer and SGD is obtained by manual subtracting of gradients.
This way first we zero our net grads:
rnn.zero_grad()

After that we get output of net and calculate our loss:
loss = criterion(output, category_tensor)
Next we calculate derivatives via backward():
loss.backward()

and finally manual subtracting:
for p in rnn.parameters():
        p.data.add_(-learning_rate, p.grad.data)

What I’ve changed in that is that I replace random data point choosing by creating train/validation datasets and add loss output.
But I can’t find the place where I spoil something.
Can you pls take another look at that?