Okay, thanks a lot for your help - I’m going to rewrite my loss and score functions.
Related question in case you know: Is the generalized dice loss the same as the dice loss when there is only one class? I recall the generalized dice loss adds weighting to each score based on the positive samples from each class.