  Multi-view representation learning captures comprehensive information from
multiple views of a shared context. Recent works intuitively apply contrastive
learning (CL) to learn representations, regarded as a pairwise manner, which is
still scalable: view-specific noise is not filtered in learning view-shared
representations; the fake negative pairs, where the negative terms are actually
within the same class as the positive, and the real negative pairs are
coequally treated; and evenly measuring the similarities between terms might
interfere with optimization. Importantly, few works research the theoretical
framework of generalized self-supervised multi-view learning, especially for
more than two views. To this end, we rethink the existing multi-view learning
paradigm from the information theoretical perspective and then propose a novel
information theoretical framework for generalized multi-view learning. Guided
by it, we build a multi-view coding method with a three-tier progressive
architecture, namely Information theory-guided heuristic Progressive Multi-view
Coding (IPMC). In the distribution-tier, IPMC aligns the distribution between
views to reduce view-specific noise. In the set-tier, IPMC builds self-adjusted
pools for contrasting, which utilizes a view filter to adaptively modify the
pools. Lastly, in the instance-tier, we adopt a designed unified loss to learn
discriminative representations and reduce the gradient interference.
Theoretically and empirically, we demonstrate the superiority of IPMC over
state-of-the-art methods.

Computer Vision and Pattern Recognition;Artificial Intelligence;Machine Learning