I think the value ranges might be a problem in your use case.
Since your model trains fine using random (normalized) data, I would suggest to normalize the input data and run the training again.
Also, if that doesn’t help, you could normalize the output additionally and de-normalize it for prediction.
The actual target signal seems to be a small signal on top of a mean value.