Solved by albanD in post #8 
              
              
                If you want to learn the hidden layer initial state, starting from 0: 
class SequenceModel(nn.Module):

    def __init__(self):
        super().__init__()
        self.lstm = nn.LSTM(input_size = 3, hidden_size = 3, bidirectional=False)
        self.hidden = nn.ParameterList((nn.Parameter(torch.zero…