Changing version 1.6 adam.py with version 1.2 adam.py copy paste did not work for me.

image737×75 3.26 KB

I also found some other differences during forward part.
Here in this code, the l_pos.grad_fn is different with two versions.
torch==1.2.0   → l_pos ==  <AsStridedBackward object at 0x7f0ed25cc990>
torch==1.6.0 → l_pos ==  <ViewBackward object at 0x7fa1a422f2d0>
Could this some kind of grad_fn change affect the backpropagation of the same code?