Many thanks for putting me into right direction. Yes, when the batch_size=1, my true_masks.shape =[1,1,640,959], when the batch_size=2, my true_masks.shape=[2,1,640,959]. Using torch.squeeze(true_masks,1) solved my problem. @ptrblck