@lopuhin after about ~200 iterations (batch size 1) my output images only have one color (= are classified to have one one homogenous segmentation label) which does not change with different images. I tried cuda and cpu mode (same problem) I also get this when only using 1 u-net layer (but not with only one standard convolutional layer). So not sure if this can be a bug. I guess I need to try out your implementation to find bugs in my code. So big thanks for sharing!