Hi,
I was training an encoder-decoder model where the encoder model has only a GRU, and the decoder is a simple MLP which is given the hidden state of GRU as the input. My data is stock market prices (Open, High, Low, Close). The inputs to the encoder are sequences of OHLC prices (say for example 20 OHLCs where 20 is the window size). The sequence in each window is independent of other windows. During the training of the model, after some iterations, all the gradients become zero (gradients of the encoder) and the hidden states become all the same. I initialize the hidden state to zero before feeding the input to the encoder for each input. Here is my encoder model:
class EncoderRNN(nn.Module):
    def __init__(self, input_size, hidden_size, device):
        super(EncoderRNN, self).__init__()
        self.device = device
        self.hidden_size = hidden_size
        self.gru = nn.GRU(input_size, hidden_size)

    def forward(self, x):
        if len(x.shape) < 3:
            x = x.unsqueeze(1)

        hidden = self.initHidden(x.shape[1])

        output, hidden = self.gru(x, hidden)
        return output, hidden

    def initHidden(self, batch_size):
        return torch.zeros(1, batch_size, self.hidden_size, device=self.device)


I feed data to the model in batches, and x.shape[1] shows the batch size.
I would be appreciated if someone can help me