  Distribution alignment can be used to learn invariant representations with
applications in fairness and robustness. Most prior works resort to adversarial
alignment methods but the resulting minimax problems are unstable and
challenging to optimize. Non-adversarial likelihood-based approaches either
require model invertibility, impose constraints on the latent prior, or lack a
generic framework for alignment. To overcome these limitations, we propose a
non-adversarial VAE-based alignment method that can be applied to any model
pipeline. We develop a set of alignment upper bounds (including a noisy bound)
that have VAE-like objectives but with a different perspective. We carefully
compare our method to prior VAE-based alignment approaches both theoretically
and empirically. Finally, we demonstrate that our novel alignment losses can
replace adversarial losses in standard invariant representation learning
pipelines without modifying the original architectures -- thereby significantly
broadening the applicability of non-adversarial alignment methods.

Machine Learning;Machine Learning