However, I’m wondering why you’re getting a high loss for patches containing just background.
If your model overfits to the background class, these patches should be nearly perfectly classified.

Thanks for answering, this is what I dont understand as well hence me asking here.
I am using the model provided here: https://github.com/wolny/pytorch-3dunet 4. I am only using their provided implementation of the U-NET model along with their implementation of DiceLoss and Dicecoefficient, I have my own dataloader class for my dataset and my own trainer.py file to train the model.
The system works as expected with the provided random data, but with my data I am getting the above results. I manually checked each input and label as well, and the inputs seem to be in order. The background only classes should not give such a high loss.
I dont know what to check for next at this stage, as I dont know what to do to fix this. =/