Solved by ptrblck in post #4 
              
              
                The issue is that the stored probs in lists are invalid after the first optimizer.step() and parameter update. 
I.e. the prob tensors were creates with a model using the initial parameter set P0. 
After the first loss.backward() and optimizer.step() operation these parameters will be updated to P1. 
…