Just re-ran and double checked it with elementwise_mean and it is significantly lower (so sorry for that misinformation), but it is still at least 5x higher than the normal BCE loss. I have 27 classes. As for setup, what exactly do you mean?