Solved by MFajcik1 in post #7 
              
              
                Interestingly, cloning gt really works! 
loss_s = reader_loss(log_probs_S_per_ex.unsqueeze(0), gt_start.clone())
loss_e = reader_loss(log_probs_E_per_ex.unsqueeze(0), gt_end.clone())

and after closer investigation, I have found the guilty! It is the += operator applied to offset 
Changing 
offset =…