Your model doesn’t seem to learn the underlying distribution properly.
Try to scale down the problem by using a tiny dataset (e.g. just 10 sample) and make sure your model is able to overfit it by playing around with the architecture and hyperparameters.
Once this is done, you could scale up the use case again.