Does anyone have another suggestion?
I also tried to run the same model on GoogleColab to check whether something on my device is wrong, but the performance is the same on GoogleColab.
So far I could narrow the problem down to the fact that the predicted output of the model is identical for each batch out of each epoch.
The predictions of the model is always a tensor with exclusively zeros and/or ones.
The same model (identical architecture) with exactly the same data performs ok when I use Keras. So the data does not seem to be the problem.
If I deactivate the Sigmoid activation of the last layer and use BCEWithLogitsLoss, the model does not get stuck.