Published as a conference paper at ICLR 2018 STABILIZING ADVERSARIAL NETS WITH PREDICTION METHODS
Adversarial neural networks solve many important problems in data science, but are notoriously difficult to train. These difficulties come from the fact that optimal weights for adversarial nets correspond to saddle points, and not minimizers, of the loss function. The alternating stochastic gradient methods typically used for such problems do not reliably converge to saddle points, and when convergence does happen it is often highly sensitive to learning rates. We propose a simple modification of stochastic gradient descent that stabilizes adversarial networks. We show, both in theory and practice, that the proposed method reliably converges to saddle points, and is stable with a wider range of training parameters than a non-prediction method. This makes adversarial networks less likely to "collapse," and enables faster training with larger learning rates.
NOTE:
I'm very willing to change my recommendation if I turn out to be wrong 
about the issues I'm addressing and if certain parts of the experiments are fixed.
Having said that, I do (think I) have some serious issues: 
both with the experimental evaluation and with the theoretical results.
I'm pretty sure about the experimental evaluation and less sure about the theoretical results.
THEORETICAL CLAIMS:
These are the complaints I'm not as sure about:
Theorem 1 assumes that L is convex/concave.
This is not generally true for GANs.
That's fine and it doesn't necessarily make the statement useless, but:
If we are willing to assume that L is convex/concave, 
then there already exist other algorithms that will provably converge
to a saddle point (I think). [1] contains an explanation of this.
Given that there are other algorithms with the same theoretical guarantees,
and that those algorithms don't magically make GANs work better, 
I am much less convinced about the value of your theorem.
In [0] they show that GANs trained with simultaneous gradient descent are locally asymptotically stable, 
even when L is not convex/concave. 
This seems like it makes your result a lot less interesting, though perhaps I'm wrong to think this?
Finally, I'm not totally sure you can show that simultaneous gradient descent won't converge 
as well under the assumptions you made.
If you actually can't show that, then the therom *is* useless, 
but it's also the thing I've said that I'm the least sure about.
EXPERIMENTAL EVALUATION:
Regarding the claims of being able to train with a higher learning rate:
I would consider this a useful contribution if it were shown that (by some measure of GAN 'goodness')
a high goodness was achieved faster because a higher learning rate was used.
Your experiments don't support this claim presently, because you evaluate all the models at the same step.
In fact, it seems like both evaluated Stacked GAN models get worse performance with the higher learning rate.
This calls into question the usefulness of training with a higher learning rate.
The performance is not a huge amount worse though (based on my understanding of Inception Scores),
so if it turns out that you could get that performance
in 1/10th the time then that wouldn't be so bad.
Regarding the experiment with Stacked GANs, the scores you report are lower than what they report [2].
Their reported mean score for joint training is 8.59.
Are the baseline scores you report from an independent reproduction?
Also, the model they have trained uses label information. 
Does your model use label information?
Given that your reported improvements are small, it would be nice to know what the proposed mechanism is by 
which the score is improved. 
With a score of 7.9 and a standard deviation of 0.08, presumably none of the baseline model runs
had 'stability issues', so it doesn't seem like 'more stable training' can be the answer.
Finally, papers making claims about fixing GAN stability should support those claims by solving problems
with GANs that people previously had a hard time solving (due to instability).
I don't believe this is true of CIFAR10 (especially if you're using the class information).
See [3] for an example of a paper that does this by generating 128x128 Imagenet samples with a single generator.
I didn't pay as much attention to the non-GAN experiments because
a) I don't have as much context for evaluating them, because they are a bit non-standard.
b) I had a lot of issues with the GAN experiments already and I don't think the paper should be accepted unless those are addressed.
[0] https://arxiv.org/abs/1706.04156 (Gradient Descent GAN Optimization is Locally Stable)
[1] https://arxiv.org/pdf/1705.07215.pdf (On Convergence and Stability of GANs)
[2] https://arxiv.org/abs/1612.04357 (Stacked GAN)
[3] https://openreview.net/forum?id=B1QRgziT (Spectral Regularization for GANs)
EDIT: 
As discussed below, I have slightly raised my score. 
I would raise it more if more of my suggestions were implemented (although I'm aware that the authors don't have much (any?) time for this - and that I am partially to blame for that, since I didn't respond that quickly).
I have also slightly raised my confidence.
This is because now I've had more time to think about the paper, and because the authors didn't really address a lot of my criticisms (which to me seems like evidence that some of my criticisms were correct).