I suppose you are predicting the mask using a softmax function?
I think you should pool in the channel dimension, not the batch dimension.
Assuming your output has the shape [batch, channels, w, h], you are calculating _, preds = torch.max(output, 0, keepdim=True), which calculates the maximal values in the current batch. If you only provide one image (batch=1), then of course the index 0 has the max value. 
Try using torch.max(output, 1, keepdim=True).