Current literature in machine learning holds that unaligned, self-interested agents do not learn to use an emergent communication channel. We introduce a new sender-receiver game to study emergent communication for this spectrum of partially-competitive scenarios and put special care into evaluation. We find that communication can indeed emerge in partially-competitive scenarios, and we discover three things that are tied to improving it. First, that selfish communication is proportional to cooperation, and it naturally occurs for situations that are more cooperative than competitive. Second, that stability and performance are improved by using LOLA (Foerster et al., 2018a), especially in more competitive scenarios. And third, that discrete protocols lend themselves better to learning cooperative communication than continuous ones.
This paper looks at the question of emergent communication amongst self-interested learning agents. The paper finds that "selfish" (ie. self-interested) agents can learn to communicate using a cheap talk channel as long as the objective is partially cooperative. 
The paper makes states that this is is a novel finding that contradicts the previous understanding of emergent communication in the literature (side point: at least some of the papers referenced for this claim did not at all make the claim). 
I believe there is a major miss-understanding here: As noted in the paper, self-interested agents can learn to communicate in settings in which the reward function is cooperative. Furthermore, it is also known that in 2 player zero-sum there is no incentive to learn a communication protocol. 
This clearly shows that talking about whether or not "selfish" agents can learn to communicate only ever makes sense within the context of a specific game / reward structure. 
With this in mind, the main finding, agents learn to somewhat communicate with each other in a simple toy setting, with more communication happening when the payouts are more cooperative, is not very interesting. 
This doesn't mean that there isn't a good paper to be written here, in principle. Finding simple settings in which SOTA multi-agent learning "fails", ie. doesn't find Nash policies, understanding why it fails and then finding ways to mend things is generally a good research direction. However, this would require a few things which are currently lacking from the paper: (1) clear understanding of the Nash policies for the different reward settings (2) Implementation of SOTA methods for MARL which are appropriate for this setting (3) In depth analysis of learning successes and failures, ideally in settings which have previously been studied in literature (given how task-specific this analysis necessarily is).
Regarding 2: General sum games will generally have mixed-strategies as Nash equilibria (just think 'rock-paper-scissors'). With this in mind, using a deterministic policy for the receiver is inappropriate for making any claims about learning in general sum games. 
Furthermore, it is well known that independent gradient descent (IGD) is not generally going to converge in general sum games (consider the loss functions X * Y and - X *Y or matching pennies). So looking at the outcome of IGD without checking for convergence means the results could be just about anything. Indeed, we don't have to go all the way to writing about emergent communication or complex "sequential social dilemma" to study this, those issues can easily be found in (iterated) matrix games. 
This gets us to the second major point of the paper. To the authors' credit,  LOLA [1] has been shown to help with convergence in general sum settings and to lead to the emergence of cooperation and reciprocity in iterated games. 
However, the key point for the ‘cooperation’ part is iterated. In a single shot setting (which is explored in this paper), there is simply no way for the agents to reciprocate with each other. So in short, I do not believe the authors' interpretation that agents learn to cooperate with each other because of LOLA, but I do believe that LOLA can help with the learning of mixed strategies (at least for the sender, given that the receiver is deterministic) and with stabilizing convergence. Lastly, the part of the experimental section is dominated by large error bars and graphs that are difficult to interpret.
Other points:
-"..but train agents to emerge their own." (and many other instances). AFAIK "to emerge something" is grammatically wrong (and also sounds really odd). 
-"Since the loss is differentiable with respect to the receiver, it is trained directly with gradient descent, so we are training in the style of a stochastic computation graph (Schulman et al., 2015).". This is a weird statement. You don't need SCGs for training a supervised objective. Also, note that the loss is also differentiable with respect to the action of the 1st agent. It is trivial in this setting to compute the true expected return, if that is what you are after. Note my point above about deterministic policies
-"We perform a hyperparameter search to over both agents’" -> spurious "to"
-"We investigate a similar scenario but concern ourselves with learning agents as opposed to fully-rational agents that have full knowledge of the structure of the game, and we do not assume that agents use an existing language, but train agents to emerge their own" .This would be interesting, if the game was complex.
- L_1 vs L - these symbols are used inconsistently, with the subscript _1 sometimes being applied and sometimes not.
-"we can look to extant results" - s/extant/extent?
-"We use the L2 metric only on hyperparameter search and keep L1 as our game’s loss to maintain a constant-sum game for the fully competitive case." - A few points: (a) the game is not in general constant sum (b) By doing this hyperparameter search the evaluation is strongly biased towards 'fair' attributions. This seems highly problematic. 
-"We report our results in Figure ??" -> Broken reference. 
-"We do not test b = 180◦ because the game is constant-sum and therefore trivially Ls1 + Lr1 = 180◦." -> So? It would still be interesting to see what learning agents do in this setting. 
[1]: "Learning with Opponent Learning Awareness", Foerster et al. 
[update: I have updated the score based on the discussion with the authors]. While the paper lacks execution and conceptual clarity, I believe the game itself is interesting and could serve as a starting point for more thorough investigation.