Concentration Inequalities for $(f,\Gamma)$-GANs
<|reference_start|>Concentration Inequalities for $(f,\Gamma)$-GANs: Generative adversarial networks (GANs) are unsupervised learning methods for training a generator distribution to produce samples that approximate those drawn from a target distribution. Many such methods can be formulated as minimization of a metric or divergence. Recent works have proven the statistical consistency of GANs that are based on integral probability metrics (IPMs), e.g., WGAN which is based on the 1-Wasserstein metric. IPMs are defined by optimizing a linear functional (difference of expectations) over a space of discriminators. A much larger class of GANs, which allow for the use of nonlinear objective functionals, can be constructed using $(f,\Gamma)$-divergences; these generalize and interpolate between IPMs and $f$-divergences (e.g., KL or $\alpha$-divergences). Instances of $(f,\Gamma)$-GANs have been shown to exhibit improved performance in a number of applications. In this work we study the statistical consistency of $(f,\Gamma)$-GANs for general $f$ and $\Gamma$. Specifically, we derive finite-sample concentration inequalities. These derivations require novel arguments due to nonlinearity of the objective functional. We demonstrate that our new results reduce to the known results for IPM-GANs in the appropriate limit while also significantly extending the domain of applicability of this theory.<|reference_end|>
@article{birrell2024concentration,
    title={Concentration Inequalities for $(f,\Gamma)$-GANs},
    author={Jeremiah Birrell},
    journal={arXiv preprint arXiv:2406.16834},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.16834},
    primaryClass={stat.ML cs.LG}
}