Do you mean like the same framework (like a DeepLabv3 or PSPNet) pre-trained on different datasets, followed by fine-tuning all of them on a shared dataset? For instance if you pre-trained on Pascal VOC 2012 with one model, and MS COCO Stuff with another, then they have different classes, and hence different network head weight shapes, but if you want to only use the classes (20 or so) from Pascal VOC, then you could do this, testing on either Pascal VOC or MS COCO stuff filtering the categories of the latter to be the desired 20.
Then there’s the question of how you combine the predictions. I guess you could simply weight the logits from each model at inference time, and use an argmax operation computed pixel-wise across classes.