Learning heavy-tailed distributions with Wasserstein-proximal-regularized $\alpha$-divergences
<|reference_start|>Learning heavy-tailed distributions with Wasserstein-proximal-regularized $\alpha$-divergences: In this paper, we propose Wasserstein proximals of $\alpha$-divergences as suitable objective functionals for learning heavy-tailed distributions in a stable manner. First, we provide sufficient, and in some cases necessary, relations among data dimension, $\alpha$, and the decay rate of data distributions for the Wasserstein-proximal-regularized divergence to be finite. Finite-sample convergence rates for the estimation in the case of the Wasserstein-1 proximal divergences are then provided under certain tail conditions. Numerical experiments demonstrate stable learning of heavy-tailed distributions -- even those without first or second moment -- without any explicit knowledge of the tail behavior, using suitable generative models such as GANs and flow-based models related to our proposed Wasserstein-proximal-regularized $\alpha$-divergences. Heuristically, $\alpha$-divergences handle the heavy tails and Wasserstein proximals allow non-absolute continuity between distributions and control the velocities of flow-based algorithms as they learn the target distribution deep into the tails.<|reference_end|>
@article{chen2024learning,
    title={Learning heavy-tailed distributions with
  Wasserstein-proximal-regularized $\alpha$-divergences},
    author={Ziyu Chen, Hyemin Gu, Markos A. Katsoulakis, Luc Rey-Bellet, Wei Zhu},
    journal={arXiv preprint arXiv:2405.13962},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.13962},
    primaryClass={stat.ML cs.LG}
}