Solved by rishikhan in post #2 
              
              
                I have found the problem. I needed to move the guard up. Here is the body of the constructor that fixes this. 
 {
       torch::NoGradGuard guard;
       emb.ptr()->weight.normal_(0, 0.01);
       emb.ptr()->weight[0] = 0;
       output.ptr()->bias.zero_();
       output.ptr()->bias += bias_offset;
…