@pyTorch_beginner, there is a chance that your problem is in training.
You can implement both forward methods in the same way:
def forward(self, x, h):
        out, h = self.lstm(x, h)
        out = self.fc(self.relu(out[:,-1]))
        return out, h

And afterwards, when calling method,
               output, h = net(inputs, h)
Usually, in LSTM, duo to presence of cell state, we change hidden to something like this:
h = tuple([each.data for each in h])
but for GRU:
h = h.data