Thanks for responding. My bad, I should’ve included info regarding my data. My dataset consists of only a sensor’s measurements which are always non-zero. My data loader basically normalizes it by dividing each measurement by the maximum_value/2. I also tried the maximum value but, it didn’t work.
I agree with your concern regarding the ReLU in decoder, I used a machine translation implementation as the baseline and it was there in that implementation, though might be impractical here. I removed that line and still have the same problem. I also updated the question with this information.
Edit: I forgot to add, any data tensor dataloader gives has the shape of [batch_size, 144], 144 meaning 144 hours of data, I split them by 72 and 72 on train and test runs.