Solved by vdw in post #2 
              
              
                You need to reset/initialize your hidden state before each forward pass, in this step you can adjust it to your current batch size. The most common pattern is to have a method in your model class like: 
def init_hidden(self, batch_size):
    if self.rnn_type == 'gru':
        return torch.zeros(self…