  Training semantic segmentation models on multiple datasets has sparked a lot
of recent interest in the computer vision community. This interest has been
motivated by expensive annotations and a desire to achieve proficiency across
multiple visual domains. However, established datasets have mutually
incompatible labels which disrupt principled inference in the wild. We address
this issue by automatic construction of universal taxonomies through iterative
dataset integration. Our method detects subset-superset relationships between
dataset-specific labels, and supports learning of sub-class logits by treating
super-classes as partial labels. We present experiments on collections of
standard datasets and demonstrate competitive generalization performance with
respect to previous work.

Computer Vision and Pattern Recognition