Every parameter seem to be in the parameter pool…
The problem seems to be the loss.backward() and optimizer.step() calls that are not updating the weights.
While debugging I did:
a = list(self.parameters())[0]
loss.backward()
self.optimizer.step()
b = list(self.parameters())[0]
torch.equal(a.data, b.data)

And it returned True so the parameter is not being updated at all. I don’t have a clue of what may be wrong with my code anymore. Here’s a small reproducible example of my code:
https://gist.github.com/miguelvr/251903423fed3b64d320b53a847aaa75 148