LEARNING WHEN TO COMMUNICATE AT SCALE IN MULTIAGENT COOPERATIVE AND COMPETITIVE TASKS
Learning when to communicate and doing that effectively is essential in multi-agent tasks. Recent works show that continuous communication allows efficient training with back-propagation in multiagent scenarios, but have been restricted to fullycooperative tasks. In this paper, we present Individualized Controlled Continuous Communication Model (IC3Net) which has better training efficiency than simple continuous communication model, and can be applied to semi-cooperative and competitive settings along with the cooperative settings. IC3Net controls continuous communication with a gating mechanism and uses individualized rewards for each agent to gain better performance and scalability while fixing credit assignment issues. Using variety of tasks including StarCraft BroodWars TM explore and combat scenarios, we show that our network yields improved performance and convergence rates than the baselines as the scale increases. Our results convey that IC3Net agents learn when to communicate based on the scenario and profitability.
This work is an extension to the work of Sukbaatar et al. (2016) with two main differences:
1) Selective communication: agents are able to decide whether they want to communicate.
2) Individualized reward: Agents receive individual rewards; therefore, agents are aware of their contribution towards the goal.
These two new extensions enable their model to work in either cooperative or a mix of competitive and competitive/collaborative settings. The authors also claim these two extensions enable their model to converge faster and better. 
The paper is well written, easy to follow, and everything has been explained quite well. The experiments are competent in the sense that the authors ran their model in four different environments (predator and prey, traffic junction, StarCraft explore, and StarCraft combat). The comparison between their model with three baselines was extensive; they reported the mean and variance over different runs. I have some concerns regarding their method and the experiments which are brought up in the following:
 
Method:
In a non-fully-cooperative environment, sharing hidden state entirely as the only option for communicate is not very reasonable; I think something like sending a message is a better option and  more realistic (e.g., something like the work of Mordatch & Abbeel, 2017)
Experiment:
The experiment "StarCraft explore" is similar to predator-prey; therefore, instead of explaining StarCraft explore, I would like to see how the model works in StarCraft combat. Right now, the authors explain a bit about the model performance in Starcraft combat, but I found the explanation confusing.
 
Authors provide 3 baselines:
1) no communication, but IR
2) no communication, no IR
3) global communication, no IR (commNet)
I think having a baseline that has global communication with IR can show the effect of selective communication better. 
There are some questions in the experiment section that have not been addressed very well. For example:
 Is there any difference between the results of table 1, if we look at the cooperative setup? 
Does their model outperform a model which has global communication with IR? 
Why do IRIC and IC work worst in the medium in comparison to hard in TJ in table1? 
Why is CommNet work worse than IRIC and IC in table 2?