The model wasn’t trained on a binary segmentation use case, but for 20 classes as stated in the docs:

The pre-trained models have been trained on a subset of COCO train2017, on the 20 categories that are present in the Pascal VOC dataset. You can see more information on how the subset has been selected in  references/segmentation/coco_utils.py .

Most likely you will get an activation containing the logits for these 20 classes (in dim1).
If you are dealing with a binary segmentation use case, you could set num_classes=2 (for a binary multi-class segmentation using nn.CrossEntropyLoss) or num_classes=1 (for a binary segmentation using nn.BCEWithLogitsLoss) in the instantiation of the model.