Nash equilibrium has long been a desired solution concept in multi-player games, especially for those on continuous strategy spaces, which have attracted a rapidly growing amount of interests due to advances in research applications such as the generative adversarial networks. Despite the fact that several deep learning based approaches are designed to obtain pure strategy Nash equilibrium, it is rather luxurious to assume the existence of such an equilibrium. In this paper, we present a new method to approximate mixed strategy Nash equilibria in multi-player continuous games, which always exist and include the pure ones as a special case. We remedy the pure strategy weakness by adopting the pushforward measure technique to represent a mixed strategy in continuous spaces. That allows us to generalize the Gradient-based Nikaido-Isoda (GNI) function to measure the distance between the players' joint strategy profile and a Nash equilibrium. Applying the gradient descent algorithm, our approach is shown to converge to a stationary Nash equilibrium under the convexity assumption on payoff functions, the same popular setting as in previous studies. In numerical experiments, our method consistently and significantly outperforms recent works on approximating Nash equilibrium for quadratic games, general blotto games, and GAMUT games.
This paper proposes a novel algorithm for finding mixed-strategy Nash equilibria in games with continuous action spaces. The paper proves convergence to a stationary Nash equilibrium under the assumption of convex utility functions. In quadratic games, general blotto games and GAMUT games, the proposed algorithm (MC-GNI) significantly outperforms its competitors. This experimental finding is not surprising, as its competitors were designed to compute pure-strategy Nash-equilibria.
This paper investigates an important, relatively unexplored problem. Its proposed algorithm appears theoretically justified and to achieve nontrivial performance in practice. However, I feel that there are some issues with this paper. 
There has been a significant amount of recent work on differentiable games. It would benefit readers for the paper to discuss how this work and those works are related. In particular, the latter are motivated by designing gradient-based algorithms for finding parameters of neural networks corresponding to local equilibria. These neural networks may themselves be playing nondeterministic strategies in some underlying game (e.g., GANs). This closely resembles the paradigm of the algorithm proposed in this paper, which finds deterministic parameters for a network that itself induces a nondeterministic strategy. This leads to a number of questions:
1. In the experiments, the paper’s language suggests that SGA is constrained to pure strategies. Why is it not parameterized by a network producing a stochastic strategy? This seems like the sensible comparison.
2. How are the convergence guarantees discussed in this paper related to the capacity of the network being used to approximate the policy? Are they lost if the network has insufficient capacity?
3. The paper mentions that SGA outperforms other existing algorithms applicable to continuous game settings. What about stable opponent shaping??
The writing in the paper would benefit from revision. While the paper generally gets the point across, much of it feels sloppy. There are also many typos, some of which are listed below.
 
Typos:
- Raghunathan et al. (2019) introduces 
- In precise
- drown
- for $\forall$
- we prepare each distribution πj a corresponding pushforward function
- Therefore, a common algorithm of computing
- On the one hand, global infimum can
- We develop a solution significantly improves the status-quo.
- of Mixed strategy on Continuous game
Other:
- The paper alternates between his and it when referring to players
Overall, I think this work is an outline for a nice paper but would like to see 
1) Clarification regarding its relationship to existing work
2) Experimental baselines in which SGA and stable opponent shaping are parameterized to produce nondeterministic strategies 
3) Cleaner writing