What's in a Latent? Leveraging Diffusion Latent Space for Domain
  Generalization
Domain Generalization aims to develop models that can generalize to novel and
unseen data distributions. In this work, we study how model architectures and
pre-training objectives impact feature richness and propose a method to
effectively leverage them for domain generalization. Specifically, given a
pre-trained feature space, we first discover latent domain structures, referred
to as pseudo-domains, that capture domain-specific variations in an
unsupervised manner. Next, we augment existing classifiers with these
complementary pseudo-domain representations making them more amenable to
diverse unseen test domains. We analyze how different pre-training feature
spaces differ in the domain-specific variances they capture. Our empirical
studies reveal that features from diffusion models excel at separating domains
in the absence of explicit domain labels and capture nuanced domain-specific
information. On 5 datasets, we show that our very simple framework improves
generalization to unseen domains by a maximum test accuracy improvement of over
4% compared to the standard baseline Empirical Risk Minimization (ERM).
Crucially, our method outperforms most algorithms that access domain labels
during training.
67cf9d9ebe9366d1ae96de5e
Xavier Thomas
claimed_verified
2025-03-12T08:42:42.540Z
6505b787dacc94cd6ce9d256
/avatars/9490807f97afc39aeb663d343a900ff9.svg
Thomas
user
XThomasBU
67cf9d9ebe9366d1ae96de5f
Deepti Ghadiyaram
What's in a Latent? Leveraging Diffusion Latent Space for Domain
  Generalization
Domain Generalization aims to develop models that can generalize to novel and
unseen data distributions. In this work, we study how model architectures and
pre-training objectives impact feature richness and propose a method to
effectively leverage them for domain generalization. Specifically, given a
pre-trained feature space, we first discover latent domain structures, referred
to as pseudo-domains, that capture domain-specific variations in an
unsupervised manner. Next, we augment existing classifiers with these
complementary pseudo-domain representations making them more amenable to
diverse unseen test domains. We analyze how different pre-training feature
spaces differ in the domain-specific variances they capture. Our empirical
studies reveal that features from diffusion models excel at separating domains
in the absence of explicit domain labels and capture nuanced domain-specific
information. On 5 datasets, we show that our very simple framework improves
generalization to unseen domains by a maximum test accuracy improvement of over
4% compared to the standard baseline Empirical Risk Minimization (ERM).
Crucially, our method outperforms most algorithms that access domain labels
during training.
A method to improve domain generalization by leveraging latent domain structures (pseudo-domains) from pre-trained feature spaces, especially diffusion models, enhances classifier performance on unseen domains.
domain generalization model architectures pre-training objectives feature richness latent domain structures pseudo-domains domain-specific variations diffusion models empirical risk minimization (ERM)