I realized that my classes are very highly imbalanced and the model was behaving as expected by classifying always as the dominant class, that’s why the errors were converging to the observed proportion of cases in the minority class. After resampling (undersample majority class, oversample minority class) and adding weights to the loss function to correct persisting imbalances my results now look like this:

Thanks for your time @ptrblck