Generative modeling of high dimensional data like images is a notoriously difficult and ill-defined problem. In particular, how to evaluate a learned generative model is unclear. In this paper, we argue that adversarial learning, pioneered with generative adversarial networks (GANs), provides an interesting framework to implicitly define more meaningful task losses for unsupervised tasks, such as for generating "visually realistic" images. By relating GANs and structured prediction under the framework of statistical decision theory, we put into light links between recent advances in structured prediction theory and the choice of the divergence in GANs. We argue that the insights about the notions of "hard" and "easy" to learn losses can be analogously extended to adversarial divergences. We also discuss the attractive properties of parametric adversarial divergences for generative modeling, and perform experiments to show the importance of choosing a divergence that reflects the final task.
This paper introduces a family of "parametric adversarial divergences" and argue that they have advantages over other divergences in generative modelling, specially for structured outputs. 
There's clear value in having good inductive biases (e.g. expressed in the form of the discriminator architecture) when defining divergences for practical applications. However, I think that the paper would be much more valuable if its focus shifted from presenting a new notion of divergence to deep-diving into the effect of inductive biases and presenting more specific results (theoretical and / or empirical) in structured prediction or other problems.  In its current form the paper doesn't seem particularly strong for either the divergence or GAN literatures. Some reasons below:
* There are no specific results on properties of the divergences, or axioms that justify them. I think that presenting a very all-encompassing formulation without a strong foundation does not add value. 
* There's abundant literature on f-divergences which show that there's a 1-1 relationship between divergences and optimal (Bayes) risks of classification problems (e.g. Reid at al. Information, Divergence and Risk for Binary Experiments in JMLR and Garcia-Garcia et al. Divergences and Risks for Multiclass Experiments in COLT).  This disproves the point that the authors make that it's not possible to encode information about the final task in the divergence. If the loss for the task is proper, then it's well known how to construct a divergence which coincides with the optimal risk.
* The divergences presented in this work are different from the above since the risk is minimised over a parametric class instead of over the whole set of integrable functions. However, practical estimators of f-divergences also reduce the optimization space (e.g. unit ball in a RKHS as in Nguyen et al.  Estimating Divergence Functionals and the
Likelihood Ratio by Convex Risk Minimization or Ruderman et al. Tighter Variational Representations of f-Divergences via Restriction to Probability Measures). So, given the lack of strong foundation for the formulation, "parametric adversarial divergences" feel more like estimators of other divergences than a relevant new family.
* There are many estimators for f-divergences (like the ones cited above and many others based e.g. on nearest-neighbors) that are sample-based and thus correspond to the "implicit" case that the authors discuss. They don't necessarily need to use the dual form. So table 1 and the first part of Section 3.1 are not accurate.
* The experiments are few and too specific, specially given that the paper presents a very general framework. The first experiment just shows that Wasserstein GANs don't perform well in an specific dataset and use that to validate a point about those GANs not being good for high dimensions due to their sample complexity. That feels like confirmation bias and also does not really say anything about the parametric adversarial GANs, which are the focus of the paper.
In summary, I like the authors idea to explore the restriction of the function class of dual representations to produce useful-in-practice divergences, but the paper feels a bit middle of the road. The theory is not strong and the experiments don't necessary support the intuitive claims made in the paper.