Hi KFrank,
Thank you four your answer, I fixed all the points you have mentioned, and I removed the last nn.ReLU()  in my model, and tried to put alternatively Tanh and ReLU for the remaining activation functions. It seems that in fact the model returns the same prediction for every input of my batches during the training phase, as shown in the following test with batches of batch_size = 10 :

pb628×546 16.4 KB

At least I have positive values, but I still can’t see what is the problem here, I even tried to print out the weights of the model for each linear transform, or avoid using ReLU but the problem still holds.
Thank you