Alright, I believe here’s the problem. In ur model, if I assume the input to CNN is of shape [B, 1, L], then the CNN outputs a tensor of shape [B, C, L] where B is the batch size, L is the sequence length, C is the channel. You then fed it into LSTM hoping it would learn the temporal information of dimension 2(L). But what’s actually happening is that, according to here 62, the LSTM by default assumes the input is of shape [L, B, INPUT_SIZE], so in this case, the LSTM is trying to learn the temporal between dimension 0(B), which apparently are independent with each other. That’s why the network is acting as a random number generator.
To solve this problem, u would have to transpose ur tensor accordingly to match the expected input shape of the LSTM and get the last hidden state according to the output shape of LSTM.