Because batchnorm is not the best normalization technique for your task. Batch norm tries to normalize all your images in a batch to same mean and std, whereas in case of segmentation you want different statistics for each image. Use InstanceNorm instead.