Solved by user_123454321 in post #7 
              
              
                Ok…I think it is the self.hidden that is causing the problem…I am not sure why you are passing self.hidden in self.lstm, do you want the lstm layer to remember the state it was in the previous batch? If so you can do a detach and store in the model like 
    x = self.embedding(x)
    x = F.relu(self…