Hmmm, although the loss may seem small, but since the background class much more abundant, the model might have got away with predicting every pixel as background. Can you try using dice as both loss and as a metric ? Also, is the loss around 0.14 for validation too ?