Solved by mrshenli in post #2 
              
              
                I believe you need to call zero_grad() either before backward() or after step(). Otherwise, the grad is always 0 when optimizer step tries to use it.