In that case i think just swapping the last layer in the classifier might be sufficient:
model = models.segmentation.fcn_resnet101(pretrained=True)

nb_classes = 2
model.classifier[4] = nn.Conv2d(
    in_channels=512,
    out_channels=nb_classes,
    kernel_size=1,
    stride=1)

Let me know, if that would work for you or if you need to dig into the model.