Based on the predictions it seems your model learns the “mean prediction” for all targets.
I’ve came across a similar behavior in another model trying to predict keypoints in images.
The “solution” was to play around with some hyperparameters as well as the model architecture.
I would generally recommend to try to overfit a small dataset (e.g. just 10 samples) first and make sure your model is able to do so. Once this is done, you could scale up the use case again by using more data.